[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] Mirrors, P2P, and easier file distribution
Hi everyone, I've been working on something tentatively called Metalink that stores the many ways to get a certain file (ftp, http, P2P) in a sort of universal download container for easy one click file distribution. This is mainly useful for large multi-located files and for multi-threaded (segmented/accelerated) downloads with download managers. I think it could be good for free software/open source community files that are stored on lots of mirrors. Things like Linux distribution ISOs & OpenOffice.org. A few download managers plan to implement this. I'm no XML expert, so I'd appreciate any comments. thank you... ant ----------------------------------------------- http://home.comcast.net/~albryan/metalink/ What is a Metalink? Metalinks bundle the various ways (FTP/HTTP/P2P) to acquire a file into one format. This can increase download performance, reliability, usability, & efficiency. A Metalink is a Multi Method Metalinker (MMM or M3talinker) document. MMM is a dialect of XML and MMM files must conform to the XML 1.0 specification. MMM might also look like WWW mirrored. Metalinks (MMM .metalink files) use XML to catalog the various ways (P2P, FTP, HTTP) to get a file/URL. I see this as especially useful for multi-threaded (segmented) downloads of large files that are mirrored (& integrated with P2P downloads) & also to list alternate addresses for web pages (for instance during a slashdotting). Benefits over traditional methods * Standard unified format that collects links for automatic accelerated (segmented) downloads from multiple sources. * Automatic load balancing distributes traffic so individual servers are under less strain. * No Single Point of Failure (SPOF) like FTP or HTTP URLs. * Links already collected for segmented (accelerated) downloading from multiple sources and they are used automatically. * No long confusing list of possibly outdated Mirrors. * Simplifies download options (language, Operating System, location, etc). * Combines Peer-to-peer (shared bandwidth) with FTP and HTTP. * Can finish BitTorrent download even if no full seeds are shared. * No separate MD5/SHA-1 file & manual process for verification. Automatic use: click on a link for a 3 gigabyte Linux distribution ISO & all the mirror locations are in your download manager & downloading from BitTorrent peers & grabbing a portion of the file from each mirror, maxing out your downstream bandwidth. Alternate links can be used as a fallback to finish downloading a .torrent that is no longer seeded 100%. Once you have a 100% seed, you will keep sharing and help the other people's .torrent downloads finish. And take a look at the OpenOffice.org download page. It has three choices for the user. Language, 41 options to choose from, operating system, 8 options, Mirror location, 71 options. And three types of P2P (BitTorrent, Magnet, & ED2K). Language, operating system, and location can be set in the client. There is a section for MD5/SHA-1 hashes and PGP signatures for verification. MD5/SHA-1 hashes are unique to each file & could be used to search for alternate locations (instead of just size & filename). So if all links went bad (or slow), the client could use a Web/P2P search engine to find mirrors/alternate locations files for the SHA-1 sum it is looking for. An example, this time for OpenOffice.org 2.0 with a BitTorrent .torrent file embedded, magnet, & ed2k links. The final file would include all OS/language combinations, and for single click downloads, the download client will automatically grab the correct version according to options you have specified. <?xml version="1.0" encoding="UTF-8"?> <metalink version="1.9" xmlns="http://home.comcast.net/~albryan/metalink/" origin="http://www.openoffice.org/mdc/ooo2.metalink" type="static" pubdate="2005-11-08-22:07:22" refreshdate="2005-11-09-03:24:18"> <files> <file name="OpenOffice.org_2.0.0_Win32Intel_install.exe"> <identity>OpenOffice.org</identity> <version>2.0.0</version> <description>OpenOffice.org 2.0.0 - free office suite</description> <tags>OpenOffice.org, office suite, OpenDocument, open source</tags> <language>en-US</language> <os>Windows32</os> <size>78597807</size> <verification> <md5>ec048f0e09431196959c6ed7bed22cae</md5> </verification> <publisher> <name>OpenOffice.org</name> <url>http://www.openoffice.org/</url> </publisher> <license> <name>LGPL</name> <url>http://www.gnu.org/copyleft/lesser.html</url> </license> <copyright>Copyright 2000-2005 Sun Microsystems Inc.</copyright> <resources> <magnet> <url> magnet:?xt=urn:bitprint:EVYF3SWQSV4CCSK46VIT7QHM2OEDEHYL.V TYBIIBLAEUMSVPQCDUAK7DXJJH2MEK7DZHTVXY&xt=urn:ed2k:4b4fb5116c0a4b261fb7e d0d721178da&dn=OpenOffice.org_2.0.0_Win32Intel_install.exe </url> <preference>90</preference> </magnet> <ed2k> <url> ed2k://|file|OpenOffice.org_2.0.0_Win32Intel_install.exe| 78597807|4B4FB5116C0A4B261FB7ED0D721178DA|h=UD55DQJEU7EYEWL5QIYLJINNQAIQ H5P7|/ </url> <preference>90</preference> </ed2k> <bittorrent> <torrent encoding="base64"> ZDg6YW5ub3VuY2U0NTpodHRwOi8vYm9yZnQuc3R1ZGVudC51dHdlbnRlLm5sOjY5NjkvYW5u b3VuY2UxMzpjcmVhdGlvbiBkYXRlaTExMjY3Njg5MzdlNDppbmZvZDY6bGVuZ3RoaTY3NDIz MTQ4ZTQ6bmFtZTMyOk9Pb18xLjEuNV9XaW4zMkludGVsX2luc3RhbGwuemlwMTI6cGllY2Ug bGVuZ3RoaTI2MjE0NGU2OnBpZWNlczUxNjA6fKscPdm2FUu3r/I9s0dCoSXwzKdgMqd/0Rjv SQCYp0QAkXNPYq2SVapoq/A/H5OZxFJvg5O2Ej0KidtQoiuTik3nTVDa+Xh9x0KD1HSnwCsQ I/23xsnITV9xIN2vRUJiRD3HE4sY7Syw2QgaNmpIUJIFiq7gXqxnZrqlxR6nkC6p1hqiXUxv ...TRIMMED TO SAVE SPACE... 3nBjypUCJOsy/NzaYnnXJx8LL7Z1PRUCYYlP4P0pZ9DQQbmbAY9VeXX1W/1rI9k66DikYI3f rQVEqy8e2RmpN1uGlDEBb4G+KoGeVfHs9Lyvn/iHBmVJ7GCnAK1HPFXjocpe3rLSWWeNwVdo l0IOZWU= </torrent> <preference>100</preference> </bittorrent> <http> <url>http://mirrors.isc.org/pub/openoffice/stable/2.0.0/OOo_2.0.0_Win32Intel _install.exe</url> <location>US</location> <preference>80</preference> </http> <ftp> <url>ftp://ftp.ussg.iu.edu/pub/openoffice/stable/2.0.0/OOo_2.0.0_Win32Intel_ install.exe</url> <location>US</location> <preference>20</preference> </ftp> <http> <url>http://mirrors.ibiblio.org/pub/mirrors/openoffice/stable/2.0.0/OOo_2.0. 0_Win32Intel_install.exe</url> <location>US</location> <preference>20</preference> </http> <ftp> <url>ftp://openofficeorg.secsup.org/pub/software/openoffice/stable/2.0.0/OOo _2.0.0_Win32Intel_install.exe</url> <location>US</location> <preference>40</preference> </ftp> </resources> </file> </files> </metalink>
|
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|