The FileCatalyst UDP-based protocol handles data streams more efficiently than TCP-based protocols like FTP. The main reasons? Each tiny packet must be acknowledged before a new one is sent.
The time spent waiting for acknowledgment is "dead air" with no data transfer. (packets go in sequential order) The Serial Nature of TCP "Got it! Send the next!" "Broken! Send it again!" "OK, Sending!" You get the idea. Now imagine this process carried out over millions of packets, and you can start to see how this "dead air" can slow the transfer. As latency (RTT) and packet loss increase,
so will "dead air". "Umm... hello? You there? No?
OK, I'm going to try that again." Packet is lost FTP Client FTP Server Plain UDP uses a different approach which looks something like this: As soon as there is a tiny glitch in the transfer, a mechanism at the PROTOCOL LEVEL (ie. built right into TCP) tries to reduce errors.
There's a lot of clever math involved, written in a bygone era. Some of it sounds good in theory, but in the real world it simply amounts to this: The TCP Sliding Window "Whoops, there was an error, so I'm going to be extra-careful and go REALLY REALLY SLOW to try to avoid another one," "...I'll be receiving until
you finish sending!" "Hmm... no more data? I assume we're done!" "OK, Sending!" UDP Sender UDP Receiver With no acknowledgment plus the ability for packets to arrive out of order (or be dropped altogether), there is no "dead air." The data can flow as fast as the link will allow. The transmission is immune to latency, which is good. But packets can be dropped, which is bad. It gets even worse for TCP This ordered stream of packets looks a bit like this: (Oh, and I'm going to be timid about going faster again...) UDP has no sliding window. "I'm missing a packet." "Sending at a conservative speed!" packet loss FTP Client FTP Server "OK, I'm going to re-send half as much." "That's fine. I like the data arriving intact and in order." Each "blip" causes the window to close to half its previous size. And it doesn't matter if the error was a fluke that should have been easy to recover from. The sliding window becomes a transfer rate ceiling. No sliding window means no protocol-driven ceiling.
Rate is set by the client side. What does FileCatalyst bring to the mix? ( Or... If TCP is so inefficient and UDP is so great, why does nearly everyone in the world still use FTP? ) 1. Reliability UDP has no built-in reliability. You simply can't use it for file transfer on its own.
FileCatalyst ADDS reliability features such as on-the-fly MD5 checksums as well as retry and resume facilities that ensure delivery (while still maintaining the benefits of UDP). 2. Automation Most FTP clients do not include scheduling and automation of transfers.
FileCatalyst HotFolder includes all the tools you need to filter and send files based on time intervals or filesystem activity.
(bonus: command-line and API tools also make it easy to slip into your existing automation processes) The Bottom Line FTP is ready to go for file transfer. UDP is not. To use UDP for file transfer, you need to build your own mechanism.
FileCatalyst was created as a superior UDP-based file transfer method. Here's what FileCatalyst provides: 3. Security FileCatalyst offers SSL encryption for login, and commands, plus AES encryption for data.
Beyond encryption, FileCatalyst offers brute force attack prevention, IP filtering, LDAP(S) and AD(S) authentication, and optional gatekeeping via an HTTP tunnel. 4. Accountability Every piece of transactional data is logged. View file transfer details in the raw logs, or use FileCatalyst Central to view current and historical records.
FileCatalyst Webmail and Workflow are web-based applications which also collect and store meta-data related to the transfers for a complete picture of every transaction. 5. Speed It's has to be mentioned one more time. FTP, HTTP, CIFS, or other traditional file transfer methods.... they will never reach full line speed if there is latency or packet loss on the link.
THis can't be fixed with a newer machine or a faster link, because the protocol itself is the bottleneck.
Only with a UDP-based system like FileCatalyst can you transfer files across your modern broadband connection at full speed. Contact FileCatalyst: 30-day trial: http://filecatalyst.com/trial
email: email@example.com (...lost packets are just ignored) 1. retransmission of lost packets (udp, but with numbered packets for retransmission)
2. congestion control (speed up a bit, slow down a bit as a continuous process)
3. block sizes for huge data and ultra-fast links Core improvements (dropped packets are completely ignored)See the full transcript