<pre> What is happening when a downloading file finishes? First of all some notes: "!!xxx" means an option or other data which is stored in MLDonkeys ini files, for example: !!temp_directory, which can be found in downloads.ini. Well, how does it all begin? Donkey============================================================================================= A timer is created in donkeyMain.ml that checks every !!compute_md4_delay whether a file is complete: add_session_option_timer enabler compute_md4_delay (fun _ -> DonkeyOneFile.check_files_downloaded (); DonkeyShare.check_shared_files () ); DonkeyOneFile.check_files_downloaded () - goes through all downloading files and calls donkeyOneFile.check_file_downloaded donkeyOneFile.check_file_downloaded - checks with the swarmer if all chunks have status 3 (complete and verified) - checks with the swarmer if the downloaded data matches the file size - calls donkeyOneFile.download_finished donkeyOneFile.download_finished - calls donkeyOneFile.declare_completed_file donkeyOneFile.declare_completed_file - among other things most important calls commonInteractive.file_completed commonInteractive.file_completed - works only on files with state FileDownloading - file is moved from !!files to !!done_files (these are sections in files.ini) - file receives state FileDownloaded - CommonShared.new_shared is called - file completion mail is send CommonShared.new_shared - puts the file into hashtable dirnames - iters through all network modules and calls CommonNetwork.network_share CommonNetwork.network_share - calls op_network_share in all network modules donkeyShare.ml let _ = network.op_network_share <- ( The following code is called on op_network_share. The donkey module has its own list of shared files besides commonShared.ml. This is a bit confusing and should be changed in the future, shared_files_new.ini is maintained only by donkeyShare.ml, shared_files.ini is maintained by Gnutella/G2/Fasttrack(GGF)-module. Here update_shared_num is called which is again in commonShared: commonShared.update_shared_num - adds the file in shareds_by_num shareds_by_num is a hashtable in commonShared which is, for example, used by the "upstats" command to display the list of shared files: driverCommands.ml "upstats", Arg_none (fun o -> let list = ref  in shared_iter (fun s -> A list named "list" is created which is filled with the results of shared_iter which gives the results of shareds_by_num: commonShared.shared_iter f = H.iter f shareds_by_num shareds_by_num is a so called "weak hashtable" which means it is only available from the module where it is created, here it is commonShared.ml. It therefore can not be accessed directly from driverCommands.ml, so shared_iter is used for this. The next command in the timer in donkeyMain.ml is "DonkeyShare.check_shared_files ()" but it works only on committed files. Committing means *moving* the physical file from the !!temp_directory to the incoming directory. This will be explained later on BitTorrent========================================================================================== Another timer is created in bTMain.ml add_session_timer enabler 60.0 (fun timer -> BTClients.recover_files (); ); bTClients.recover_files () - calls check_finished bTClients.check_finished - checks with the swarmer if the file is finished, then calls bTClients.download_finished if Int64Swarmer.check_finished swarmer then download_finished file bTClients.download_finished - notifies the tracker that the file is finished - calls commonInteractive.file_completed (see notes in the Donkey section) op_network_share - does nothing here. The finished file(s) are still in !!temp_directory waiting for commit ==================================================================================================== committing files ==================================================================================================== Committing takes place either automatically (!!auto_commit true, files are checked every minute) or manually, both ways call commonInteractive.file_commit Here is the manual way driverCommands.ml "commit", Arg_none (fun o -> List.iter (fun file -> file_commit file ) !!done_files; This code checks every minute if a file was moved to !!done_files and, if auto_commit is set to true, commits the file, this happens in the background driverMain.ml let minute_timer () = if !!auto_commit then List.iter (fun file -> file_commit file ) !!CommonComplexOptions.done_files commonInteractive.file_commit - here the real action of moving files takes place, an important comment from the source code: (******** These two functions 'file_commit' and 'file_cancel' should be the two only functions in mldonkey able to destroy a file, the first one by moving it, the second one by deleting it. Note that when the network specific file_commit function is called, the file has already been moved to the incoming/ directory under its new name. *) - "let incoming" seeks for the appropiate incoming directory, if the downloaded file is a directory (means multifile torrent) another incoming is chosen in respect of the sharing strategy used. The first shared directory with the corresponding strategy is chosen, so there is no need to have several shared directories with a incoming_* strategy. It does however not hurt, files to be shared can be put into those directories. (are directories created if one of the sharing strategies is missing?) - "let new_name" checks for the file name the commit file should have -> file_commited_name commonInteractive.file_commited_name - appends an ongoing number to the filename if the file already exists in incoming - "set_file_disk_name" calls commonFile.set_file_disk_name - commonFile.set_file_disk_name - really moves the files via Unix32.rename - Unix32.rename takes care if the finished file is really a directory from a multifile torrent - Unix32.rename copies the files if renaming fails due to the fact that !!temp_directory and the incoming directory are on seperate physical partitions, this can block the core if the files are huge. It is therefore advised to have !!temp_directory and the incoming directory on the same physical partition - "file_best_name" - take a look at files .ini - one option is called file_filename which is the default name for the file, normally it corresponds with the name present in the ED2K link or the "name" option in a torrent file - another option is called file_filenames, AFAIK only used by the Donkey module here alternative filesnames are stored which is BTW a good way of identifying fake files (result?) - "Unix32.destroy" - MLDonkey has a table of used file descriptors (fd), here the fd of the file is removed - next line removes the empty directory of a multifile torrent in !!temp_directory - "impl.impl_file_ops.op_file_commit impl.impl_file_val new_name" - this long line calls the commit function in the various network modules, keep in mind the file(s) is/are already moved, GGF/FileTP donÂ´t use this function - donkeyInteractive.file_ops.op_file_commit - DonkeyShare.remember_shared_info - unshare_file - bTInteractive.ml - (to be understood;-) and written) - commonShared.new_shared is called - file_state is updated to FileShared - file is removed from !!done_files and !!files (what about the swarmer for the file, they are not removed from files.ini) (what about commonSources, are the sources removed?) ======================================================= Donkey: I noticed that that finished files are present twice in shared_files_new.ini - once with their real name - second with "temp/urn_ed2k_ABABABABABABA" as file name, other data like md4s are the same This seems due to the fact that donkeyShare.remember_shared_info is called twice, once via commonInteractive.file_commit impl.impl_file_ops.op_file_commit impl.impl_file_val new_name donkeyInteractive.file_ops.op_file_commit <- (fun file new_name DonkeyShare.remember_shared_info file new_name and before committing via (to be removed?) donkeyOneFile.declare_completed_file DonkeyShare.remember_shared_info file (file_disk_name file) </pre>
Retrieved from "
What links here