MLDonkey :: View topic - Direct Connect support

FAQ

Memberlist

Usergroups

Profile

Direct Connect support
Goto page Previous 1, 2, 3, 4, 5, 6, 7 Next

MLDonkey Forum Index -> Problems with MLDonkey Client (Bittorrent, DirectConnect, Gnutella,...)

View previous topic :: View next topic

Author

Message

PeeGee
neophyte

Joined: 17 Feb 2010
Posts: 5

Posted: Fri Feb 19, 2010 9:37 pm Post subject:

Paranoik wrote:

Hi, i never used DC, but with downloads.ini files it happens sometimes too. To avoid this i do stop core. then edit files, and then start core again. Can you try the same?

Hi Paranoik,

How do I stop/restart the core?

Thanks / PG

PeeGee
neophyte

Joined: 17 Feb 2010
Posts: 5

Posted: Tue Feb 23, 2010 12:14 am Post subject:

PeeGee wrote:

Paranoik wrote:

Hi, i never used DC, but with downloads.ini files it happens sometimes too. To avoid this i do stop core. then edit files, and then start core again. Can you try the same?

Hi Paranoik,

How do I stop/restart the core?

Thanks / PG

Hi all,

I found the command to restart the core and the passwordpart of directconnect.ini works just fine after that.

I now managed to connect to the hub and I can see all the user and so on, but I'm having problems with the download. I cant manage to receive any filelist, and users on the hub cant get the filelist from mldonkey.

Any clues on this?

I also noticed that it's only possible to receive usermessages from the hub, not send. This is due to the charset-settings:

Exception Charset.CharsetError

Is there a way to change this?

Thanks / PG

PeeGee
neophyte

Joined: 17 Feb 2010
Posts: 5

Posted: Wed Mar 10, 2010 3:06 pm Post subject:

Solutions anyone? Thanks / PG

Devils
neophyte

Joined: 19 Apr 2009
Posts: 13

Posted: Sat Mar 13, 2010 10:54 am Post subject:

I did not correctly handle DC in MLDonkey. Namely, when connected to a DC-hub, another user can not see my share. Volume share see, and to connect and download can not. Is the problem?

ygrek
professional

Joined: 20 Mar 2010
Posts: 517

Posted: Sun Apr 25, 2010 9:49 am Post subject:

@PeeGee, try the version from CVS (or wait for release), probably this is fixed. Receiving filelists works for me fine, not sure about downloading my filelist. @Devils, please describe your problem with more details.

somedamnthing
neophyte

Joined: 31 Jul 2010
Posts: 22

Posted: Tue Aug 03, 2010 12:06 am Post subject:

First, thanks for the DC support. I'm using MLDonkey specifically for this, and the basic file operation is wonderful.

In the interest of making scripting easier, I have a few requests:

* clearing of dcmsglogs: something like:

Code:

dclogclear <name> | <server> <ip>'

to wipe the message buffer of a user or hub.

* open up the size max of individual messages: the only place I've found this to be the case (based on logs) is in dcProtocol.ml, lines 387 and 884. After altering both to 75000 and recompiling, I've been able to communicate with bots again.

* allow servername in place of ip for commands requiring a server argument

* make dcmsglog work with server channels inside telnet

That's about it. Thanks for your consideration.

ygrek
professional

Joined: 20 Mar 2010
Posts: 517

Posted: Wed Aug 04, 2010 9:18 pm Post subject:

Thanks for the feedback.

Quote:

In the interest of making scripting easier, I have a few requests:

Just curious - what kind of a scripting, what for?

Quote:

[...]

That's about it. Thanks for your consideration.

Sounds reasonable, wait for the patches to test :)

somedamnthing
neophyte

Joined: 31 Jul 2010
Posts: 22

Posted: Wed Aug 04, 2010 11:10 pm Post subject:

ygrek wrote:

Just curious - what kind of a scripting, what for?

Automated file searches/grabs. I'm toying between dealing with nc and curl bases in either bash or perl. curl's what I'm using now for my current tests, as the better functionality lies there.

Quote:

Sounds reasonable, wait for the patches to test Smile

Sweet!

As my mldonkey/DC implementation goes forward, I'm noticing one massive showstopper, and that's a shared_files_dc.ini limitation. mlnet-real spirals up to 100% CPU and hangs (with no log errors, connections time out) on a shared_files_dc.ini size of around 1.2 megs.

Thinking this might be an issue with TTH generation and file writes, I whipped up a perl script to generate the shared_files_dc.ini from whole cloth. This custom file is 2.5 megs in size.

When mlnet-real loads, I once again see the logs report nicely, then hanging after all services are up, presumably once it tries to build some internal table with the over 8000 files I've got listed.

I noticed Ocaml has sqlite3 bindings (ocaml-sqlite3-1.0.3). Would a transition to sqlite (or something) be easier on mlnet-real, or is there some internal mechanism to mldonkey that can be tweaked?

Edit: literally half an hour later, the service finally became responsive (http didn't time out, it just hanged until a response).

ygrek
professional

Joined: 20 Mar 2010
Posts: 517

Posted: Fri Aug 06, 2010 9:15 am Post subject:

somedamnthing wrote:

Automated file searches/grabs.

Sounds like this functionality can be useful for other users as well.. Don't mind implementing it in mldonkey core itself instead of external scripts?

somedamnthing wrote:

Let's investigate - please show the output of sysinfo, shared_files_dc.ini that causes problems, also downloads.ini and direct_connect.ini (or show which settings were changed compared to default).

spiralvoice
Sage

Joined: 06 Jan 2003
Posts: 3982
Location: Germany

Posted: Sat Aug 07, 2010 9:29 pm Post subject:

somedamnthing wrote:

Edit: literally half an hour later, the service finally became responsive (http didn't time out, it just hanged until a response).

strace´ing MLDonkey might show bottle-necks
_________________
Link overview and precompiled cores here: http://mldonkey.sourceforge.net/DownloadLinks

spiralvoice
Sage

Joined: 06 Jan 2003
Posts: 3982
Location: Germany

Posted: Sun Aug 08, 2010 6:41 pm Post subject:

somedamnthing wrote:

* open up the size max of individual messages: the only place I've found this to be the case (based on logs) is in dcProtocol.ml, lines 387 and 884. After altering both to 75000 and recompiling, I've been able to communicate with bots again.

Please test this patch: https://savannah.nongnu.org/patch/index.php?7274
_________________
Link overview and precompiled cores here: http://mldonkey.sourceforge.net/DownloadLinks

somedamnthing
neophyte

Joined: 31 Jul 2010
Posts: 22

Posted: Wed Aug 11, 2010 9:35 pm Post subject:

ygrek wrote:

somedamnthing wrote:

Automated file searches/grabs.

Sounds like this functionality can be useful for other users as well.. Don't mind implementing it in mldonkey core itself instead of external scripts?

Well, a lot of what I'm doing is specific to the bots I'm dealing with. I sure don't mind implementing hooks or whatnot to assist.

somedamnthing wrote:

Let's investigate - please show the output of sysinfo, shared_files_dc.ini that causes problems, also downloads.ini and direct_connect.ini (or show which settings were changed compared to default).

I'm using FreeBSD, so there's no sysinfo per se.

Using ktrace (instead of the suggested strace) yields interesting results. When mlnet is good, it is very very good, and when it is bad, it can't seem to properly thread. First, (generally) every 10 seconds:

Code:

48872 mlnet-real 423.765625 RET _umtx_op -1 errno 60 Operation timed out
48872 mlnet-real 423.765673 CALL gettimeofday(0xbf8fdf80,0)
48872 mlnet-real 423.765679 RET gettimeofday 0
48872 mlnet-real 423.765686 CALL clock_gettime(0,0xbf8fde84)
48872 mlnet-real 423.765691 RET clock_gettime 0
48872 mlnet-real 423.765696 CALL _umtx_op(0x28905760,0x8,0x1,0x28905740,0xbf8f
de7c)
48872 mlnet-real 430.014205 RET _umtx_op -1 errno 60 Operation timed out
48872 mlnet-real 430.014248 CALL gettimeofday(0xbf9fef80,0)
48872 mlnet-real 430.014254 RET gettimeofday 0
48872 mlnet-real 430.014261 CALL clock_gettime(0,0xbf9fc7e4)
48872 mlnet-real 430.014266 RET clock_gettime 0
48872 mlnet-real 430.014271 CALL _umtx_op(0x289055e0,0x8,0x1,0x289055c0,0xbf9f
c7dc)
48872 mlnet-real 433.765950 RET _umtx_op -1 errno 60 Operation timed out
48872 mlnet-real 433.765987 CALL gettimeofday(0xbf8fdf80,0)
48872 mlnet-real 433.765993 RET gettimeofday 0
48872 mlnet-real 433.765999 CALL clock_gettime(0,0xbf8fde84)
48872 mlnet-real 433.766032 RET clock_gettime 0
48872 mlnet-real 433.766036 CALL _umtx_op(0x28905760,0x8,0x1,0x28905740,0xbf8fde7c)

Then, hundreds of times per second:

Code:

48872 mlnet-real 437.518923 CALL mmap(0,0x100000,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,0xffffffff,0,0)
48872 mlnet-real 437.518951 RET mmap 705691648/0x2a100000
48872 mlnet-real 437.524924 CALL mmap(0,0x100000,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,0xffffffff,0,0)
48872 mlnet-real 437.524950 RET mmap 706740224/0x2a200000
48872 mlnet-real 437.530880 CALL mmap(0,0x100000,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,0xffffffff,0,0)
48872 mlnet-real 437.530907 RET mmap 708837376/0x2a400000
48872 mlnet-real 437.534330 CALL mmap(0,0x100000,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,0xffffffff,0,0)
48872 mlnet-real 437.534356 RET mmap 709885952/0x2a500000
48872 mlnet-real 437.538873 CALL mmap(0,0x100000,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,0xffffffff,0,0)
48872 mlnet-real 437.538905 RET mmap 711983104/0x2a700000
48872 mlnet-real 437.542867 CALL mmap(0,0x100000,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,0xffffffff,0,0)
48872 mlnet-real 437.542890 RET mmap 713031680/0x2a800000
48872 mlnet-real 437.604657 CALL madvise(0x2a301000,0x7e000,MADV_FREE)
48872 mlnet-real 437.604809 RET madvise 0
48872 mlnet-real 437.604815 CALL madvise(0x2a201000,0xfc000,MADV_FREE)
48872 mlnet-real 437.604932 RET madvise 0
48872 mlnet-real 437.604937 CALL madvise(0x2a17f000,0x7e000,MADV_FREE)
48872 mlnet-real 437.604998 RET madvise 0
48872 mlnet-real 437.605009 CALL munmap(0x2a200000,0x100000)
48872 mlnet-real 437.605064 RET munmap 0
48872 mlnet-real 437.605074 CALL munmap(0x2a300000,0x100000)
48872 mlnet-real 437.605137 RET munmap 0
48872 mlnet-real 437.605147 CALL munmap(0x2a400000,0x100000)
48872 mlnet-real 437.605201 RET munmap 0

umtx is the userland mutex for pthreads in FreeBSD. Should I take this to the FreeBSD port maintainer instead, or does this look like something that should be investigated source-side?

edit: after mlnet finally gets its act together, the log outputs something like:

Code:

2010/08/11 16:11:28 [cUp] Detected clock jump. deltat 1891.603362 adjusted to 10.000000

spiralvoice wrote:

Please test this patch: https://savannah.nongnu.org/patch/index.php?7274

That looks exactly like what I've got, and that portion is running brilliantly. Thanks, spiralvoice.

ygrek
professional

Joined: 20 Mar 2010
Posts: 517

Posted: Thu Aug 12, 2010 7:22 am Post subject:

Quote:

I'm using FreeBSD, so there's no sysinfo per se.

Sorry for the confusion, sysinfo is mldonkey's command, just run it in telnet or web ui.

Quote:

umtx is the userland mutex for pthreads in FreeBSD.

Probably this is the ocaml's runtime lock and some thread is not releasing it properly. Can you attach gdb when it behaves badly and show the output of thread apply all bt

Also show ldd on mlnet binary.

somedamnthing
neophyte

Joined: 31 Jul 2010
Posts: 22

Posted: Thu Aug 12, 2010 6:53 pm Post subject:

Here's all the dump info you requested. Any redacted info (user/host/dirs) is encapsulated with square brackets. Note that I didn't see this hang issue when I had a smaller (test) list of shared files.

sysinfo:

Code:

> sysinfo

--Buildinfo--
Version: MLNet Multi-Network p2p client version 3.0.3 (mlnet-real)
Networks: Global Shares Direct Connect Fasttrack FileTP BitTorrent Donkey (SUI)
Ocaml version: 3.11.1 - C compiler version: 4.2.1 - C++ compiler version: 4.2.1
Built on: FreeBSD i386 7.3-RELEASE (little endian)
Configure args: '--enable-ocamlver=3' '--with-libiconv-prefix=/usr/local' '--disable-gui' '--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/info/' '--build=i386-portbld-freebsd7.3' 'build_alias=i386-portbld-freebsd7.3' 'CC=cc' 'CFLAGS=-O2 -fno-strict-aliasing -pipe' 'LDFLAGS= -L/usr/local/lib -L/usr/local/lib' 'CPPFLAGS= -I/usr/local/include -I/usr/local/include' 'CXX=c++' 'CXXFLAGS=-O2 -fno-strict-aliasing -pipe'
Features: threads zlib-1.2.3 bzip2-1.0.5 gd(jpg/png-1.4.1) iconv(active) magic(active) no-check-bounds

--Runinfo--
MLDonkey user: admin (PW Protected) - uptime: 13h 50m 10s - running as users:[USERNAME]
Enabled nets: DirectConnect
Server usage: disabled (you are not able to connect to ED2K Servers)
Geoip: enabled, GeoLite data created by MaxMind, available from http://maxmind.com/
IP blocking: local: 0 ranges - web: 199944 ranges
Libmagic: file-type recognition database present
System info: FreeBSD [HOSTNAME] 7.3-RELEASE FreeBSD 7.3-RELEASE #0: Sun Mar 21 06:15:01 UTC 2010 root@walker.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
language: EN - locale: ASCII - UTC offset: -0700
max_string_length: 16777211 - word_size: 32 - max_array_length: 4194303 - max_int: 1073741823
max file descriptors: 11095 - max useable file size: 2^63-1 bits (do the maths ;-p)

--Portinfo--
Network | Port|Type
--------------+------+-------------------
Core | 9998|http_port
Core | 4000|telnet_port
Direct Connect| 4444|client_port TCP+UDP

--Diskinfo--
Directory |Type | used| free|%free|Filesystem
-------------------------------+-----------------------------+--------+--------+-----+----------
/usr/home/[USERNAME]/.mldonkey |core/ini files | 165.02G| 36.44G| 18%|ufs
temp |temp/downloading | 165.02G| 36.44G| 18%|ufs
/home/[USERNAME]/[DIR]/Incoming|shared (incoming_directories)| 165.02G| 36.44G| 18%|ufs
/home/[USERNAME]/[DIR]/Incoming|shared (incoming_files) | 165.02G| 36.44G| 18%|ufs
/home/[USERNAME]/[DIR]/Shared |shared (all_files) | 153.59G| 297.51G| 65%|ufs
mlnet-real_tmp |$MLDONKEY_TEMP | 165.02G| 36.44G| 18%|ufs

ldd of mlnet-real:

Code:

/usr/local/bin/mlnet-real:
libcharset.so.1 => /usr/local/lib/libcharset.so.1 (0x2839b000)
libz.so.4 => /lib/libz.so.4 (0x2839e000)
libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x283b0000)
libbz2.so.3 => /usr/lib/libbz2.so.3 (0x284a6000)
libgd.so.4 => /usr/local/lib/libgd.so.4 (0x284b7000)
libmagic.so.3 => /usr/lib/libmagic.so.3 (0x284f3000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x28505000)
libm.so.5 => /lib/libm.so.5 (0x285fa000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x2860f000)
libthr.so.3 => /lib/libthr.so.3 (0x2861a000)
libc.so.7 => /lib/libc.so.7 (0x2862f000)
libpng.so.6 => /usr/local/lib/libpng.so.6 (0x28735000)
libjpeg.so.11 => /usr/local/lib/libjpeg.so.11 (0x2875c000)
libfreetype.so.9 => /usr/local/lib/libfreetype.so.9 (0x28792000)

gdb:

Code:

(gdb) thread apply all bt

Thread 3 (Thread 0x28901040 (LWP 100272)):
#0 0x28707af0 in memmove () from /lib/libc.so.7
#1 0x28b734d4 in ?? ()
#2 0x0000000c in ?? ()
#3 0x00000000 in ?? ()
#4 0x00000001 in ?? ()
#5 0xbfbfeb78 in ?? ()
#6 0x2a700ff0 in ?? ()
#7 0x2ac4d000 in ?? ()
#8 0x2899ef38 in ?? ()
#9 0x00000000 in ?? ()
#10 0x295c4618 in ?? ()
#11 0x00000000 in ?? ()
#12 0x08344120 in std::operator+<char, std::char_traits<char>, std::allocator<char> > ()
#13 0x083440f5 in std::operator+<char, std::char_traits<char>, std::allocator<char> > ()
#14 0x08344542 in std::operator+<char, std::char_traits<char>, std::allocator<char> > ()
#15 0x0834459a in std::operator+<char, std::char_traits<char>, std::allocator<char> > ()
#16 0x0834520d in std::operator+<char, std::char_traits<char>, std::allocator<char> > ()
#17 0x082a1912 in ?? ()
#18 0x0003cdc9 in ?? ()
#19 0x00079b91 in ?? ()
#20 0x2958784c in ?? ()
---Type <return> to continue, or q <return> to quit---
#21 0x00000003 in ?? ()
#22 0x2937c950 in ?? ()
#23 0x0826a707 in ?? ()
#24 0xbfbfebac in ?? ()
#25 0x0826a66a in ?? ()
#26 0x00079b91 in ?? ()
#27 0x000f5719 in ?? ()
#28 0x2897505c in ?? ()
#29 0x29380004 in ?? ()
#30 0x2937c91c in ?? ()
#31 0x084e7bc8 in __progname ()
#32 0x28927ffc in ?? ()
#33 0x2958784c in ?? ()
#34 0x080ad4f2 in ?? ()
#35 0xbfbfebcc in ?? ()
#36 0x080ad480 in ?? ()
#37 0x2937c930 in ?? ()
#38 0x2a7d11d4 in ?? ()
#39 0x2a84de28 in ?? ()
#40 0x080ad9fc in ?? ()
#41 0x2a94666c in ?? ()
#42 0x082737f7 in ?? ()
#43 0xbfbfebe4 in ?? ()
#44 0x082737dd in ?? ()
#45 0x29345074 in ?? ()
#46 0x082734d1 in ?? ()
#47 0x28956120 in ?? ()
#48 0x08273973 in ?? ()
---Type <return> to continue, or q <return> to quit---
#49 0xbfbfec08 in ?? ()
#50 0x08273860 in ?? ()
#51 0x2891bd2c in ?? ()
#52 0x082a66ed in ?? ()
#53 0x0836a478 in __progname ()
#54 0x00000001 in ?? ()
#55 0x080512bb in ?? ()
#56 0x0804e9cd in ?? ()
#57 0x083514a6 in std::operator+<char, std::char_traits<char>, std::allocator<char> > ()
Previous frame inner to this frame (corrupt stack?)

Thread 2 (Thread 0x28901150 (LWP 100313)):
#0 0x2862b217 in __error () from /lib/libthr.so.3
#1 0x2862adf8 in __error () from /lib/libthr.so.3
#2 0x289055e0 in ?? ()
#3 0x00000008 in ?? ()
#4 0x00000001 in ?? ()
#5 0x289055c0 in ?? ()
#6 0xbf9fc7dc in ?? ()
#7 0x00000000 in ?? ()
#8 0x00000000 in ?? ()
#9 0x2862974f in pthread_setcancelstate () from /lib/libthr.so.3
#10 0x28628fed in pthread_cond_signal () from /lib/libthr.so.3
#11 0x08337b2b in std::operator+<char, std::char_traits<char>, std::allocator<char> > ()
#12 0x2862073f in pthread_getprio () from /lib/libthr.so.3
#13 0x00000000 in ?? ()

---Type <return> to continue, or q <return> to quit---
Thread 1 (Thread 0x28901260 (LWP 100075)):
#0 0x2862b217 in __error () from /lib/libthr.so.3
#1 0x2862adf8 in __error () from /lib/libthr.so.3
#2 0x28905760 in ?? ()
#3 0x00000008 in ?? ()
#4 0x00000001 in ?? ()
#5 0x28905740 in ?? ()
#6 0xbf8fde7c in ?? ()
#7 0xbf8fde58 in ?? ()
#8 0x00000030 in ?? ()
#9 0x2862974f in pthread_setcancelstate () from /lib/libthr.so.3
#10 0x28628fed in pthread_cond_signal () from /lib/libthr.so.3
#11 0x0832fa9f in std::operator+<char, std::char_traits<char>, std::allocator<char> > ()
#12 0x2862073f in pthread_getprio () from /lib/libthr.so.3
#13 0x00000000 in ?? ()
(gdb)

spiralvoice
Sage

Joined: 06 Jan 2003
Posts: 3982
Location: Germany

Posted: Thu Aug 12, 2010 8:14 pm Post subject:

somedamnthing wrote:

I'm using FreeBSD

Afaics this patch is not included in MLDonkey. Maybe it helps?
_________________
Link overview and precompiled cores here: http://mldonkey.sourceforge.net/DownloadLinks

Display posts from previous:

	MLDonkey Forum Index -> Problems with MLDonkey Client (Bittorrent, DirectConnect, Gnutella,...)	All times are GMT Goto page Previous 1, 2, 3, 4, 5, 6, 7 Next
Page 3 of 7

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum