From steve@ecf.toronto.edu Wed May 17 18:49:23 EDT 1995 Article: 27196 of comp.protocols.tcp-ip Xref: cannon.ecf comp.protocols.tcp-ip:27196 comp.unix.programmer:29729 Newsgroups: comp.protocols.tcp-ip,comp.unix.programmer Path: cannon.ecf!steve From: steve@ecf.toronto.edu (Steve Kotsopoulos) Subject: Announce: beta release of the Adaptive File Distribution Protocol X-Nntp-Posting-Host: stealth.ecf Message-ID: Followup-To: comp.protocols.tcp-ip Summary: looking for AFDP beta testers Keywords: group communication, reliable, multicast, broadcast, publishing Sender: news@ecf.toronto.edu (News Administrator) Organization: University of Toronto, Engineering Computing Facility Date: Wed, 17 May 1995 22:41:43 GMT The Adaptive File Distribution Protocol (AFDP) began as a reliable multicast transport protocol. Since IP multicast is not yet available under many operating systems, AFDP is also capable of using broadcast or unicast if necessary. The source code for AFDP is now available via anonymous ftp from: ftp.ecf.toronto.edu:/pub/afdp/ If you would like to try the first public beta release of AFDP please read the README file which is appended below. AFDP README =========== Introduction ------------ The Adaptive File Distribution Protocol (AFDP) addresses the need for efficient and reliable group communication. By exploiting the one-to-many transmission capabilities of multicast and broadcast, it is possible to send a file to multiple hosts simultaneously, using less bandwidth and thus obtaining greater efficiency than repeated unicasting. While performance measurements indicate the superiority of multicast, AFDP dynamically selects from available transmission modes so as to maximize efficiency and throughput while guaranteeing delivery of files to all hosts. The protocol, built on top of UDP, is based on the publishing model. Any machine receiving files is called a subscriber while any machine sending files is called a publisher. One special subscriber is designated as the secretary; this machine is responsible for managing group membership and authorizing publishers. A rate-based flow control mechanism is used to provide reliable file distribution at a transfer rate that the slowest host can accommodate. You can call AFDP from any of your applications, or just use the existing file transfer mechanisms of AFDP. Code Portability ---------------- AFDP has been compiled and tested on the following Unix platforms: SGI IRIX4.x, IRIX5.x Sun SunOS 4.1.3, Solaris 2.3 Mips Risc/OS 4.5.2 Digital Ultrix 3.1 (little-endian Microvax III) PC (486) Consensys SVR4.2 We tried to maximize portability by implementing AFDP top-down, and testing it on a wide variety of systems at each step. Thus, the code has very few machine-specific directives; most #ifdef statements are used to turn on or off optional features such as multicast support. Here is an outline of the changes we required for a few ports: SGI one change, to include Risc/OS one #ifdef, because network MTU is smaller by 8 bytes i386 SVR4.2 added one #include to a .h file, and 4 libraries to Makefile We expect that porting to other systems should be relatively painless. Obtaining AFDP -------------- The source code for AFDP is available via anonymous ftp from: ftp.ecf.toronto.edu:/pub/afdp/ Anyone wishing to join the beta-testers mailing list should contact afdp-beta-request@ecf.toronto.edu. Please include your preliminary experiences compiling and running the software. Documentation ------------- Online manual pages are in the 'man' directory of the source tree. More information is available on the AFDP web page, url http://www.ecf.toronto.edu/afdp/ including "Exploiting Group Communications for Reliable High Volume Data Distribution" by Jeremy Cooperstock and Steve Kotsopoulos. To appear in Proc. of IEEE PACRIM, Pacific Rim Conference on Communications, Computers, Visualization and Signal Processing, Victoria, May 1995. Building AFDP ------------- To build AFDP, first edit the Makefile to configure the system for your particular installation. You will have to define target directories for the executable binaries (BINDIR), shell scripts (SHDIR), and man pages (MANDIR), and may need to specify system-dependent compilation flags (CFLAGS) and libraries (LIBS). Once this is done, type "make install" to compile and install the programs. Use "make install.man" to install the manual pages. For the Impatient Explorer -------------------------- If you want to get an idea of how AFDP works without reading all of the documentation, type "ajoin" on any number of machines which will act as subscribers, and then "asend file1 file2" on the machine which will act as the publisher. For more detailed debugging messages, add more '-d' options. If the files are gif images, the remote end will display them automatically with magic-cat. Components ---------- The two main components of AFDP are afdpjoin and afdpsend. Both afdpjoin and afdpsend programs must select the same group in order for them to work together. Multiple groups may coexist, as may multiple publishers in the same group. Usage of the two programs is as follows: afdpjoin [-DVdhr] [-a aclfile] [-g group] [-p port] [-s secretary_host] tuning options: -I [Internet mode] -T threshold -t multicast_ttl afdpsend [-DVdhr] [-a aclfile] [-g group] [-p port] [-s secretary_host] [-c command] file ... tuning options: -C cache_size -M [disable multicasting] -N [nice mode] -R [rabid mode] -S stepsize -b broadcast_size -i forced_interval -u unicast_size -w window_size Utility shell scripts are provided to run these programs more easily. The first, ajoin, creates a temporary directory for file storage, then runs afdpjoin with some useful default arguments: afdpjoin -t 2 -d $* The second, asend, simply runs afdpsend with common arguments: afdpsend -d -c magic-cat $* The '-c magic-cat' option instructs all subscribers to run the program magic-cat on any received files. Magic-cat is a shell script we provide that interprets files logically, based on their contents. For example, 'ghostview' is used to display PostScript files, and 'xv' is used to display images. Magic-cat is very useful for displaying multimedia data. Typical LAN Usage ----------------- Usually, you start up groups by just running 'ajoin' on as many hosts as you want. Then, to send files to the group, run 'asend file ...'. The '-D' argument to afdpjoin is very useful for starting up a large number of subscribers on different hosts, because it allows you to run it without a controlling terminal (ie. from a script file). The use of this flag is discouraged on the secretary, since it also disables the 'Q' command used to shutdown the group. A more flexible AFDP shutdown facility is being worked on. Using Internet Mode ------------------- For AFDP to work on an internetwork it cannot rely on broadcast at all, neither to locate a secretary, nor for file transmissions. Furthermore, even if the hosts support multicast, multicast cannot be used if the routers in between the hosts don't forward multicast packets correctly. We recommend the following technique to use AFDP on Internet: 1. Select a secretary host, say stealth.ecf. 2. On stealth.ecf, run 'afdpjoin -I' to startup a secretary in Internet mode. 3. Startup additional subscribers with 'afdpjoin -I -s stealth.ecf'. 4. Publish files to the group with 'afdpsend -M file ...' You can drop the '-M' argument if your routers support multicast, or if 'afdpsend -V' reports "[without multicast support]". Finally ------- This version of AFDP is a beta-test release, so please do not re-distribute it or make it available via ftp or www servers. Questions, bug reports, complaints and suggestions should be sent to afdp-beta@ecf.toronto.edu. Accolades should be posted to the net. acl.c and acl.h by Craig Metz . authuser.c and authuser.h by Dan Bernstein, and in public domain. utils.c and utils.h by John DiMarco . All other files in this distribution, unless explicitly marked otherwise, are: Copyright (c) 1994, 1995 by Jeremy Cooperstock and Steve Kotsopoulos, All Rights Reserved. You may use this software for research and educational purposes. Companies or institutions wishing to make use of AFDP for commercial purposes should contact the authors . -- Jeremy Cooperstock Steve Kotsopoulos -- Steve Kotsopoulos P.Eng. steve@ecf.toronto.edu Systems Analyst, Engineering Computing Facility, University of Toronto http://www.ecf.toronto.edu/ecf/staff/steve/