在网络状况不好的情况下,对于文件的传输,我们希望能够支持可以每次传部分数据。首先从文件传输协议FTP和TFTP开始分析, FTP是基于TCP的,一般情况下建立两个连接,一个负责指令,一个负责数据;而TFTP是基于UDP的,由于UDP传输是不可靠的,虽然传输速度很快,但对于普通的文件像PDF这种,少了一个字节都不行。本次以IM中的文件下载场景为例,解析基于TCP的文件断点续传的原理,并用代码实现。 什么是断点续传? 断点续传其实正如字面意思,就是在下载的断开点继续开始传输,不用再从头开始。所以理解断点续传的核心后,发现其实和很简单,关键就在于对传输中断点的把握,我就自己的理解画了一个简单的示意图:
原理: 断点续传的关键是断点,所以在制定传输协议的时候要设计好,如上图,我自定义了一个交互协议,每次下载请求都会带上下载的起始点,这样就可以支持从断点下载了,其实HTTP里的断点续传也是这个原理,在HTTP的头里有个可选的字段RANGE,表示下载的范围,下面是我用Java语言实现的下载断点续传示例。 提供下载的服务端代码:
[java] view plaincopy import java.io.File; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.io.RandomAccessFile; import java.io.StringWriter; import java.net.ServerSocket; import java.net.Socket; public class FTPServer { class Sender extends Thread{ private InputStream in; private OutputStream out; private String filename; public Sender(String filename, Socket socket){ try { this.out = socket.getOutputStream(); this.in = socket.getInputStream(); this.filename = filename; } catch (IOException e) { e.printStackTrace(); } } @Override public void run() { try { System.out.println("start to download file!"); int temp = 0; StringWriter sw = new StringWriter(); while((temp = in.read()) != 0){ sw.write(temp); } String cmds = sw.toString(); System.out.println("cmd : " + cmds); if("get".equals(cmds)){ File file = new File(this.filename); RandomAccessFile access = new RandomAccessFile(file,"r"); StringWriter sw1 = new StringWriter(); while((temp = in.read()) != 0){ sw1.write(temp); sw1.flush(); } System.out.println(sw1.toString()); int startIndex = 0; if(!sw1.toString().isEmpty()){ startIndex = Integer.parseInt(sw1.toString()); } long length = file.length(); byte[] filelength = String.valueOf(length).getBytes(); out.write(filelength); out.write(0); out.flush(); System.out.println("file length : " + length); byte[] buffer = new byte[1024*10]; int tatol = (int) length; System.out.println("startIndex : " + startIndex); access.skipBytes(startIndex); while (true) { if(tatol == 0){ break; } int len = tatol - startIndex; if(len > buffer.length){ len = buffer.length; } int rlength = access.read(buffer,0,len); tatol -= rlength; if(rlength > 0){ out.write(buffer,0,rlength); out.flush(); } else { break; } } out.close(); in.close(); access.close(); } } catch (IOException e) { e.printStackTrace(); } super.run(); } } public void run(String filename, Socket socket){ new Sender(filename,socket).start(); } public static void main(String[] args) throws Exception { ServerSocket server = new ServerSocket(8888); String filename = "E:\\ceshi\\mm.pdf"; for(;;){ Socket socket = server.accept(); new FTPServer().run(filename, socket); } } }
下载的客户端代码:
[java] view plaincopy import java.io.File; import java.io.InputStream; import java.io.OutputStream; import java.io.RandomAccessFile; import java.io.StringWriter; import java.net.InetSocketAddress; import java.net.Socket; public class FTPClient { public void Get(String filepath) throws Exception { Socket socket = new Socket(); socket.connect(new InetSocketAddress("127.0.0.1", 8888)); OutputStream out = socket.getOutputStream(); InputStream in = socket.getInputStream(); byte[] cmd = "get".getBytes(); out.write(cmd); out.write(0); int startIndex = 0; File file = new File(filepath); if(file.exists()){ startIndex = (int) file.length(); } System.out.println("Client startIndex : " + startIndex); RandomAccessFile access = new RandomAccessFile(file,"rw"); out.write(String.valueOf(startIndex).getBytes()); out.write(0); out.flush(); int temp = 0; StringWriter sw = new StringWriter(); while((temp = in.read()) != 0){ sw.write(temp); sw.flush(); } int length = Integer.parseInt(sw.toString()); System.out.println("Client fileLength : " + length); byte[] buffer = new byte[1024*10]; int tatol = length - startIndex; access.skipBytes(startIndex); while (true) { if (tatol == 0) { break; } int len = tatol; if (len > buffer.length) { len = buffer.length; } int rlength = in.read(buffer, 0, len); tatol -= rlength; if (rlength > 0) { access.write(buffer, 0, rlength); } else { break; } System.out.println("finish : " + ((float)(length -tatol) / length) *100 + " %"); } System.out.println("finished!"); access.close(); out.close(); in.close(); } public static void main(String[] args) { FTPClient client = new FTPClient(); try { client.Get("E:\\ceshi\\test\\mm.pdf"); } catch (Exception e) { e.printStackTrace(); } } }
测试 原文件、下载中途断开的文件和从断点下载后的文件分别从左至右如下:
断点前的传输进度如下(中途省略): Client fileLength : 51086228 finish : 0.020044541 % finish : 0.040089082 % finish : 0.060133625 % finish : 0.07430574 % finish : 0.080178164 % ... finish : 60.41171 % finish : 60.421593 % finish : 60.428936 % finish : 60.448982 % finish : 60.454338 % 断开的点计算:30883840 / 51086228 = 0.604543361471119 * 100% = 60.45433614% 从断点后开始传的进度(中途省略): Client startIndex : 30883840 Client fileLength : 51086228 finish : 60.474377 % finish : 60.494423 % finish : 60.51447 % finish : 60.53451 % finish : 60.554558 % ... finish : 99.922035 % finish : 99.942085 % finish : 99.95677 % finish : 99.96213 % finish : 99.98217 % finish : 100.0 % finished!
断点处前后的百分比计算如下: ============================下面是从断点开始的进度==============================
本方案是基于TCP,在本方案设计之初,我还探索了一下介于TCP与UDP之间的一个协议:UDT(基于UDP的可靠传输协议)。 我基于Netty写了相关的测试代码,用Wireshark拆包发现的确是UDP的包,而且是要建立连接的,与UDP不同的是需要建立连接,所说UDT的传输性能比TCP好,传输的可靠性比UDP好,属于两者的一个平衡的选择,感兴的可以深入研究一下。
|