Older article, but still worth some comments:

- Not too many ISPs will block UDP completely. More to the point, though, two enterprises will likely have their own servers at a colo and free reign to use whatever protocol they choose. I've encountered situations in which UDP seemed to be blocked wholesale, but this has been exceedingly rare and is more often tied to the corporate firewall than the ISP. Easily solved by IT.

- TCP window inefficiency is not a Windows-only artifact. It's built into the protocol.

- It's definitely proprietary! :) UDP is being used at the transport layer, but when your own back channel is built into the entire process, with proprietary commands being exchanged, it's more than raw UDP. Raw UDP would not work at all for large or international file transfer unless you were insanely lucky.

- There is indeed reliability built into it. Lost packets are recovered periodically, and MD5 checksums may be performed on the fly (at the block level) or at the end of the file transfer. Proper resume and retry (unlike FTP which we all know can trash a large transfer and require restart) add reliability as well.

- It's not all about "it's data going across by UDP," which as stated earlier is not new. But it IS an excellent way to get full bandwidth with a software-only solution. All of FileCatalyst's top competitors in the sphere are also using this method; it's not magic. But where the "magic" comes in is how it's wrapped up and how it's presented to the end user. The other features in the applications are the differentiators.

Case in point, as Mark says, FileCatalyst has a series of 3 applets, and the uni-directional ones are also controllable with a Javascript API.

--

At the time this article was "current", the whitepaper(s?) may have been a bit lean on technical info. There's a better one available now, and also a 4-part series on our blog talking about many of the other features that go into an acceleration app.

Whew! Apologies for the long post! Hopefully someone will dig it out of the archive and read it some day. ;-)

I actually investigated this a it more and spoke to one of the FileCatalyst sales reps. They claimed that they have a Java applet that loads in the web browser so there is actually no install on client side (since most people have Java already). So assunming the firewall or NAT is not an issue, this could be a very nice alternative to transfer files on a WAN where is too slow.

this idea requires both parties (transmitting and receiving) to be using Filecatalyst...

So hardly going to be widely adopted.

Isn't this a new Application using the existing UDP protocol?

If it were a new protocol, then drivers and network stacks and what-not would need to be updated to handle it.

This is an application layer handling of errors and retransmits, right?

I can't find an RFC for FileCatalyst (r) to read up on and the white papers I saw were pretty lean from a technical standpoint.

What they are doing is using UDP as it was intended. Fast, throw-away, send n' pray packets. Although UDP does have per packet checksums, there is no reliability, so they are using their app to handle it.
That's how you use it.

Wrong. In fact, it wastes bandwidth like hell, and latency is even more troublesome. I'd like to see this new protocol on top of BitTorrent, and NO JAVA PLEASE!


Bimodem or Hyperlink were the choices before... ymodem and xmodem sucked. zmodem when no better option was available.

No, not every packet must be acknowledged one at a time. And yes, increasing window can help.

But TCP still has limits as to how big a window you have. On low latency networks, you rarely see the limit -- even Gbps speeds. Replies get back quickly enough.

On high latency (think across-ocean or satellite transfers), you exhaust that window real quickly. And then the protocol ends up sitting there waiting for replies. TCP never kicks into high gear. 

The more bandwidth you have, the more evident it is. For most people on DLS/Cable modems -- the line speed is slow enough that you can still come close to saturating your connection. 

But think large enterprises leasing OC3 fiber to get data from US to Japan. When that connection starts to behave like a T1, you're going to be upset. Often, companies find it cheaper to load an HDD onto a plane and fly it across.

Nice idea, but not new. NFS often comes in two flavors, one via TCP and one via UDP, exactly for the reason you described nicely. But, one more time I have to ask is this solution applicable? A lot of ISPs block all UDP-based protocols completely, not just in one direction (one direction is blocked by almost all phony ISPs), because they want to prohibit exactly what Filecatalyst is made for. We have the same situation at most companies, research labs and universities. Security is the usual standard disclaimer (LIE). This is not a problem with the protocol. This has more to do with what is left of the Internet, partly thanks to the MAFIAA. Here we have just one more example where interest groups like the MAFIAA repress the development of new, useful technologies.

Would you please take the time and look at
http://physics.nist.gov/cuu/Units/binary.html and the rest at http://physics.nist.gov/cuu/Units/index.html
From a 12-year old I would expect things like "GigE", "1K", "1KBps" etc., but not from a journalist who writes about IT every day.
Make up your own cheat slip if you have difficulties to remember things (nothing wrong with that!).

B - byte
bit - bit
kB - kilobyte, 10^3 bytes = 1000 bytes (notice the lower-case "k", uppercase K means Kelvin)
MB - 10^6 bytes, megabyte
GB - 10^9 bytes, gigabyte
KiB - 1024 bytes
kB/s - kilobyte per second
kbit/s kilo bit per second
mb, kb - millibarn, kilobarn (has nothing to do with bits and bytes)
kg - kilogram
km - kilometer
5 cm - distance
5*4 cm^2 - area
5*4*3 cm^3 - volume

I have EXTENSIVELY experimented with scaling options using tcp/ip under windows and trying to measure the latency. Basically for speed between 10 and 100 Mbit/s TCP window scaling is ok. 

If you happen to have a Gigabit or highter speed conneciton to the internet with a high latency, or live on mars with a extreme high latency, then the tcp scaling option fall short. But this really a very rare condition. 

All this program does is throw overboard the TCP/Ip throttling options. If enough people do this then they will push out other users and in the end will ALL have a slower speed IMHO.

leuk_he

PS windows api to the tcp/ip windows options suck in XP. 



"Congestion avoidance"

TCP has a lot of the overhead it does so that when links begin to saturate, the whole network doesn't implode. It's why extensive testing is done with new TCP implementations to make sure they play nice together. It sounds like this protocol won't play nice with anything.

So, as long as few people use this, everything will be great. As more people use it, TCP connections will slow down (slower web browsing, slower email access, etc). If it becomes as widespread as bittorrent, things will get really ugly.

Ditto the comment about TCP window size. Maybe this protocol is meant for certain antiquated OSes that don't have intelligent management of TCP window size? (*cough* Dimdows *cough*)

Personally I have done bulk transfer of a hundred or more gigabytes of data using rsync (connecting via SSH, which in turn runs over TCP) between NZ and the US, and managed to get close to line speeds. So who needs another protocol?

I have the big TCP Windowsh, Windowsh Vista.

DAAMIT Charlie!!

Woz this mean I'm to buy ALL NEW
Bits N Bobbles!

BTBy, Thnx 4 that uther tips up.

(wheeze)

You've got to be kidding!
http://en.wikipedia.org/wiki/User_Datagram_Protocol

The first thing that comes to mind is the protocol works like the Zmodem protocol quite popular during the BBS days (before the internet was popular). Instead of using a serial port layer, they just replace it with UDP. It keeps sending, the receiving end checks that each packet is error free with some form of CRC (MD5 perhaps in the new protocol) and sends back a respond only when one of the data packets was received in error.

It also just crossed my mind that this protocol may not be NAT friendly, but until a further read-up is done, I may simply be wrong about this.

... or YModem or ZModem?
.. or Kermit or Punter Protocol or Sealink?

BBS protocolz rulez, mannnnnn

I was excited until I heard that it was implemented with the devils work. i.e. Java. Unfortunately, it now won't get anywhere near my PC.

What's wrong with bittorrent? It got perfect error checking and uses bandwidth exceptionally well.

Your statement is not correct, TCP uses a window to acknowledge packets. So it's not true that each packet must be acked, it depends on how large is the windows. Simply increasing the window can help GBE file transfers.

With it's built-in error correction, it would be really nice to see this implemented by bittorrent.