[Logo] Jaikoz and SongKong Forums
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
Musicbrainz server busy  XML
Forum Index -> Jaikoz Issues Go to Page: 1, 2 Next 
Author Message
Wabbit

Pro

Joined: 10/11/2008 13:19:26
Messages: 27
Offline

I've had this happen quite frequently lately. The problem is that Jaikoz interrupts its current operation when it encounters this. When you've been waiting a long time for it to finish a batch and it stops unexpectedly somewhere in the middle, that gets pretty frustrating.

I'm letting Jaikoz process @500 - 750 files at a time, autocorrecting and then using "Save and Move" to separate tagged files from untagged ones.

any chance Jaikoz can be set to keep trying Musicbrains rather than give up?
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

Hi, it does retry five times before giving up.

However if you are finding that it fails time and time again on the same file though that would sound like a bug rather than a real connection failure.

thanks Paul (Administrator)
Wabbit

Pro

Joined: 10/11/2008 13:19:26
Messages: 27
Offline

I'd rather see it persist and wait for musicbrainz(or my network connection) to re-aquire, rather than have to start over... if it pops up that a connection is lost (for whatever reason) a CHOICE to terminate....
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

Are you based in the Europe, looking at the discussions on Musicbrainz it appears there has been alot of problems connecting to it for the last week (possibly because a large transalantic internet connection got cut) from outside US. I take your point on board but under normal circumstances the 5-retry system works fine, I found it only ever hit the 5 retry limit if either my Internet connection was completly down or Musicbrainz was down.

thanks Paul (Administrator)
Wabbit

Pro

Joined: 10/11/2008 13:19:26
Messages: 27
Offline

nope, im here in the U.S. but I think it's happening because I'm on a wireless connection to my router via usb wireless-g stick. It drops out momentarily, has done so for as long as I can remember, so persisting would overcome this...
tudor



Joined: 30/12/2008 05:54:48
Messages: 4
Offline

I am based in Switzerland and I get very often the 'network busy' error.
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

Just recently or for a while. The problem on the Musicbrainz server has now been fixed http://blog.musicbrainz.org/?p=368

thanks Paul (Administrator)
eliasnehme



Joined: 09/12/2007 12:27:25
Messages: 1
Offline

Paul:

Is it possible for the software to distiguish and make in a color the files that cause the problems, skip them and keep going?
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

No, if its a network problem then it could happen to any file, if its a file problem then normally jaikoz handles it and does continue, if it doesnt there is a bug that I dont know about that needs fixing.

thanks Paul (Administrator)
Wabbit

Pro

Joined: 10/11/2008 13:19:26
Messages: 27
Offline

what I need is for it to distinguish a network dropout and wait until its connected, then resume, or at least the option to alter the number of retries....
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

Wabbit wrote:
what I need is for it to distinguish a network dropout and wait until its connected, then resume, or at least the option to alter the number of retries.... 

This could be an option, I'll add it to the todo list. BUT I still think the solution in your case is to get a moere reliable wireless connection

thanks Paul (Administrator)
barton986



Joined: 18/11/2008 17:55:29
Messages: 7
Offline

I have been having this same issue, starting with my latest upgrade of Jaikoz version (2.8.0 b1053). I am on a wired connection in the US.
It does not follow a particular song or file. I am able to select the files that did not get processed and restart and eventually finish.
Hope this adds some light to the issue.
-Jonathan
chroma

Pro

Joined: 25/08/2008 14:50:33
Messages: 39
Offline

fwiw, i've been seeing similar behavior. very reliable wired us connection.
pschonmann

Pro
[Avatar]
Joined: 30/11/2008 14:02:22
Messages: 239
Offline

I have this problem occasionally too. And i think its not problem of my line.

Support Jaikoz! I did it!
[Email] [WWW]
pschonmann

Pro
[Avatar]
Joined: 30/11/2008 14:02:22
Messages: 239
Offline

ive got some wireshark packets, maybe usefull

Code:
 GET /ws/1/track/?type=xml&limit=10&query=track%3A%22the+beckoning%22++AND+%28artist%3A%22konflict%22+release%3A%22drum+and+bass+armageddon+%5C%28disc+1%5C%29%22+tnum%3A%224%22+%29+++AND+qdur%3A%28236+237+238+239+240+%29 HTTP/1.1
 
 User-Agent: Java/1.6.0_11
 Host: musicbrainz.org
 Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
 Connection: keep-alive
 
 HTTP/1.1 503 Service Temporarily Unavailable
 Date: Wed, 14 Jan 2009 20:23:38 GMT
 Server: Apache/1.3.41 (Unix) mod_perl/1.30
 X-Rate-Limited: 600.5 600.0 10
 
 Content-Type: text/plain; charset=utf-8
 Via: 1.1 musicbrainz.org
 Connection: close
 Transfer-Encoding: chunked
 
 
 48
 The MusicBrainz web server is currently busy.
 Please try again later.
 
 0
 


I think i exceed limit 600 request per/minute ( X-Rate-Limited: 600.5 600.0 10)

Support Jaikoz! I did it!
[Email] [WWW]
locutus

Pro

Joined: 31/01/2008 04:44:27
Messages: 50
Offline

Same problem here, independend from connection type. So the problem is not solved.
Wabbit

Pro

Joined: 10/11/2008 13:19:26
Messages: 27
Offline

what i'd like to see when it pops up "musicbrainz server busy" is a countdown to re-establish the connection and continue (without losing it's current place) with the OPTION to quit.

as it stands now, my only choice is to restart tagging from the beginning.
Even though Im only doing 2-3000 files at a time, this adds many extra hours having to restart constantly.

This ISN't a trivial problem.....
chroma

Pro

Joined: 25/08/2008 14:50:33
Messages: 39
Offline

i also did a dump and saw a 503 before seeing it posted here, after yet another case of this. i like the countdown to re-establish with option to quit idea.

i suggest that in order to make the experience as pleasant as possible for the users and as operationally reasonable for musicbrainz, you implement an exponential back-off on the countdown timer, starting with a low value.

thanks
kieran



Joined: 21/01/2009 23:51:43
Messages: 5
Offline

I have this same problem, Musicbrainz server busy
It only happens when I do Austocorrect using Autocorrector, if I just do Autocorrect from Musicbrainz server it works fine.

I am using a wired connection with no issues
I am in Australia
I am using Mac OS X 10.5.6 and the latest java that Apple update sends me, Jaikoz 2.8.1


BTW this is very cool software,
Thanks heaps
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

kieran wrote:
I have this same problem, Musicbrainz server busy
It only happens when I do Austocorrect using Autocorrector, if I just do Autocorrect from Musicbrainz server it works fine
 

Really, thats interesting could you send me your logs please (using Action/Create Support Files)

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

I've implemented a 10 retry mechanism with increasing delays between each retry and a few other adjustments and tested it against 10,000 files without it failing so this should solve the problem, will get this release rolled out within two weeks.

If anyone is still running into problems I will add the option to retry indefinitently.

thanks Paul (Administrator)
pschonmann

Pro
[Avatar]
Joined: 30/11/2008 14:02:22
Messages: 239
Offline

Maybe usefull is check http header

X-Rate-Limited: 600.5 600.0 10, where 600.5 means actual request, 600 is limit, 10 is 10 second. If you reach 600+ requests per 10 seconds youll be temporatily "banned" you can test it with multiple opened links from firefox for example.
-- Edit.
In firefox when you reach limit ( 10 for 10 secs youll get

Your requests are exceeding the allowable rate limit (11.8, limit is 11.0 per 10 seconds)
Please slow down then try again.


Support Jaikoz! I did it!
[Email] [WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

Jaikoz has inbuilt throttler which prevents more than one query a second, and Im sure its not brokem.

thanks Paul (Administrator)
pschonmann

Pro
[Avatar]
Joined: 30/11/2008 14:02:22
Messages: 239
Offline

And is some testcase how to track this error ? What you want track paul ? some headers or jaikoz debug files ?

Support Jaikoz! I did it!
[Email] [WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

Fix now available in Jaikoz 2.8.2

thanks Paul (Administrator)
christian

Enthusiast

Joined: 31/08/2008 20:37:36
Messages: 16
Offline

Paul,
sorry, no joy here. Issue still exists with BM Server busy.

Here is the spacing between requests that failed:
Feb 10, 2009 12:15:24 AM: WARNING: MiniProcess 0 having problems connecting to MusicBrainz Service, may be down or just busy, retry attempt 1
Feb 10, 2009 12:15:25 AM: WARNING: MiniProcess 0 having problems connecting to MusicBrainz Service, may be down or just busy, retry attempt 2
Feb 10, 2009 12:15:27 AM: WARNING: MiniProcess 0 having problems connecting to MusicBrainz Service, may be down or just busy, retry attempt 3
Feb 10, 2009 12:15:30 AM: WARNING: MiniProcess 0 having problems connecting to MusicBrainz Service, may be down or just busy, retry attempt 4
Feb 10, 2009 12:15:34 AM: WARNING: MiniProcess 0 having problems connecting to MusicBrainz Service, may be down or just busy, retry attempt 5
Feb 10, 2009 12:15:39 AM: WARNING: MiniProcess 0 having problems connecting to MusicBrainz Service, may be down or just busy, retry attempt 6
Feb 10, 2009 12:15:45 AM: WARNING: MiniProcess 0 having problems connecting to MusicBrainz Service, may be down or just busy, retry attempt 7
Feb 10, 2009 12:15:52 AM: WARNING: MiniProcess 0 having problems connecting to MusicBrainz Service, may be down or just busy, retry attempt 8
Feb 10, 2009 12:16:00 AM: WARNING: MiniProcess 0 having problems connecting to MusicBrainz Service, may be down or just busy, retry attempt 9

I thought there were supposed to have a more spaced out starting interval. This starts at 1 second and then increments by one (your new algo I guess). But sometimes when it failed in the past requests were spaced by 5 sec or 3 sec or 7 sec (has always done this). Is this design intent to make initial retries a random starting interval?

I still think the issue is environmental or an uninitialized (or corrupted) variable. It can fail consitantly on a specific file in one period of time but allowing it to sit for say an hour and then retrying will often have that file succeed. Other files around it have no problem at either point in time.

Is there a simple way to trace the exact request and response between Jaikoz and MB? This would at least allow us to isolate whether it is local or an MB interaction.

Christian
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

christian wrote:

I thought there were supposed to have a more spaced out starting interval. This starts at 1 second and then increments by one (your new algo I guess). But sometimes when it failed in the past requests were spaced by 5 sec or 3 sec or 7 sec (has always done this). Is this design intent to make initial retries a random starting interval?
 

Yes previously there was only a 1 second delay between each retry.


christian wrote:

Is there a simple way to trace the exact request and response between Jaikoz and MB? This would at least allow us to isolate whether it is local or an MB interaction.
 

If you set logging to -l7 it will log all calls made to Musicbrainz in the debug log, perhaps you could try this and send me the logs with Action/Create Support FIles

I also need to take this up futher with Musicbrainz, because we rarely used to hit this issue.

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7298
Offline

pschonmann wrote:
Maybe usefull is check http header

X-Rate-Limited: 600.5 600.0 10, where 600.5 means actual request, 600 is limit, 10 is 10 second. If you reach 600+ requests per 10 seconds youll be temporatily "banned" you can test it with multiple opened links from firefox for example.
 

Sounds like you could be right this information is now logged in my dev server and Im taking to Musicbrainz about this.

thanks Paul (Administrator)
christian

Enthusiast

Joined: 31/08/2008 20:37:36
Messages: 16
Offline

I think I figured it out!

A number of users are going through an http proxy and cache either in their own environment or their ISP's environment.

When MB fails with 503 (service unavailable) the proxy caches this as a "negative hit" and gives it a Time To Live (TTL) before it will pass a new request. In my case I use squid which defaults to a TTL of 5 minutes.

So once you hit a 503, the cache will continue to fail your requests within the TTL window.

You should be able to override this in the HTTP request within Jaikoz as in your case your subsequent requests are in fact meant to bypass any caches.

I verified this by waiting for a 503 with retries then disabled the proxy while it was retrying which then allowed Jaikoz to proceed properly.

If I disable the proxy all together and run Jaikoz I get the occasional MB busy message but it works fine on the first retry.

Given many people won't have control over their cache and proxy, it is best to have Jaikoz force the proxy not to cache.

If you wish, I can test a fix as I do have control on my cache.

Christian
christian

Enthusiast

Joined: 31/08/2008 20:37:36
Messages: 16
Offline

I set the negative_ttl=0 on squid.

Just ran >5000 files and it completely successfully.

MB returned 503 a number of times and the most retried I needed was 2.

 
Forum Index -> Jaikoz Issues Go to Page: 1, 2 Next 
Go to:   
Powered by JForum 2.1.6 © JForum Team