[Logo] Jaikoz and SongKong Forums
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
Maximum Memory Size  XML
Forum Index -> Jaikoz Issues
Author Message
OneTrueDabe


[Avatar]

Joined: 15/05/2008 13:24:20
Messages: 6
Offline

I have a 8-Core Mac Pro w/ 4GB of RAM, and upwards of 86,500 songs in my iTunes Library.

Unfortunately, loading these into Jaikoz (obviously) blows out the JVM heap.

I edited "/Applications/Jaikoz.app/Contents/Info.plist" and bumped the "-Xmx" size up as high as I could -- which was about "-Xmx=2124m" -- but that still wasn't enough to load my entire library.

What I ended up having to do was request the JVM to operate in 64-bit mode. Again, in "Info.plist":

Code:
     <key>JVMArchs</key>
     <array>
          <string>x86_64</string>
     </array> 
 


(I'm told that will degrade back to 32-bit if no 64-bit architecture is available.)

Question: Will this have any impact (positive OR negative) on Jaikoz itself?

I searched the Forum but didn't see any mention, one way or the other, of using a 64-bit JVM. (And I'm posting this in case others run into the same issue.)

Cheers!

--
:- Dabe
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7487
Offline

Ive not done any work with the 64bit JVM yet, so let us know how you get on.
However I wouldn't recommend loading so many tracks in one go, it means Jaikoz has alot of work to do just managing the indexes ecetera.

thanks Paul (Administrator)
OneTrueDabe


[Avatar]

Joined: 15/05/2008 13:24:20
Messages: 6
Offline

paultaylor wrote:
Ive not done any work with the 64bit JVM yet, so let us know how you get on. 


Altogether, I'd say 64-bit was a non-issue -- things ran well.

paultaylor wrote:
However I wouldn't recommend loading so many tracks in one go, it means Jaikoz has alot of work to do just managing the indexes ecetera. 


Yeah, I guess I came to the same conclusion... In the end, I decided it was easier to deal with my collection in ~20k batches: 0-D, E-J, K-R, and S-Z.

Each batch took about ~5 minutes to load, ~9 hours to "Autocorrect Tags from MusicBrainz", and another ~5 minutes to save.

--
:- Dabe
Nocturnal

Pro

Joined: 05/06/2007 16:41:06
Messages: 20
Offline

I find that Jaikoz usually runs out of memory not because of too many files. That just makes Jaikoz really slow because it constantly has to search through and sort the list of loaded files. The problem of not enough memory lies with the mipcore.exe program that extracts the audio from compressed form to wav-form for analysis. When you have an audio file of one hour (a long DJ set, for example), that means 44100 Hz x 16 bit x 2 channels x 3600 seconds = 606MB

This comes on top of all your running programs, the operating system, and Jaikoz itself (which is also using a lot of memory for the file list and all the ID3 information, even more so when you have a lot of cover art).

One thing you can do to reduce the memory usage of Jaikoz, is disable the column that displays the cover art.

I'm currently doing a batch of 60 000 files. The loading of the files went fine. 30 000 of these already have an acoustic ID saved, so for the first step (analysis with genpuid, acoustic analysis) this leaves 30 000 files.

This is on a computer with 3GB of RAM. Windows XP and the JRE are 32 bit. Because everything is 32 bit, the maximum heap I can assign to Java is around 1600MB (currently using -Xmx1612m). This is not a bug in Jaikoz, but a limitation of the 32 bit memory addressing system. For more information for Windows on this limitation, see http://support.microsoft.com/kb/924054.
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7487
Offline

Thanks for the info, its worth pointing out that if your OS is 64bit such as Windows Vista or OSX Leopard there is also a 64bit version of Java freely available.

thanks Paul (Administrator)
mcsetsr



Joined: 11/06/2008 09:51:04
Messages: 2
Offline

64 JRE 1.6_06 Vista 64 and still no luck with setting the max memory past 2048MB on command line and can not use batch file to load app. With the 2048MB setting I can load 25-26k songs before running out. Seems the application uses double memory when running the 64 jre even at startup it used 122MB without load of 1 song.

If I change this to 4096 it loads a couple soungs then says out of memory.

"C:\Program Files (x86)\Jthink\Jaikoz\jaikoz.exe" -Jmaxheap=2048000000

Also seems if you have some corrupt music files...they use tons of memory. Jaikoz does tell you it is corrupt, so at least I have been able to delete those from my library
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7487
Offline

mcsetsr wrote:

Also seems if you have some corrupt music files...they use tons of memory. Jaikoz does tell you it is corrupt, so at least I have been able to delete those from my library  

Maybe you are hitting another corrupt file which it is failing to load, as I stated in another post if you keep stalling at the same point despite increasing your memory try missing out the problem folder. If you have any of these corrupt files that you have identified as increasing the memory load in Jaikoz in your recycle bin perhaps you could send me so I can indentify further.

Im going to write a simple memory analysis test that we can use to identify memory problems when loading specific files.

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7487
Offline

Have found a bug in Java which can occur when loading large PNG images, the net resuklts oif this is Jaikoz can give an OutOfMemoryError when it hasnt actually run out of heap memory.

For the technically minded more details here:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6687968

Ive done a fix/workaround for Jaikoz 2.5.0 Beta 2/Final which will be available early next week.

thanks Paul (Administrator)
heartspeace



Joined: 05/06/2010 18:24:02
Messages: 2
Offline

Is this still true or have you gone to be able to do 64 bit so we can really use our machines? (8 cores 12 gigs here).

paultaylor wrote:
Ive not done any work with the 64bit JVM yet, so let us know how you get on.
However I wouldn't recommend loading so many tracks in one go, it means Jaikoz has alot of work to do just managing the indexes ecetera. 
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7487
Offline

Jaikoz does work on 64bit JVM

thanks Paul (Administrator)
MAzE5h1p69wB

Pro

Joined: 21/04/2010 14:05:13
Messages: 83
Offline

Does it work with memory?

Because for some reason 3.7.1 keeps crashing when I try to specifically tell Java to please do feel free to use the resources in my computer, make yourself at home and work you piece of shit virtual machine.

Code:
 @echo off
 set CLASSPATH=
 java -jar lib\jaikoz.jar -l2 -m2 -f -Xms150m -Xmx3200m -XX:MaxPermSize=150m
 


Code:
 06/08/2010 10.47.46:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
 LoadingFiles:WARNING: Memory low:init = 16777216(16384K) used = 233072576(227609
 K) committed = 259522560(253440K) max = 259522560(253440K)
 06/08/2010 10.47.47:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
 LoadingFiles:WARNING: Memory low after Garbage Collect:init = 16777216(16384K) u
 sed = 180555400(176323K) committed = 259522560(253440K) max = 259522560(253440K)
 
 06/08/2010 10.47.47:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
 LoadingFiles:WARNING: Memory low:init = 16777216(16384K) used = 246557536(240778
 K) committed = 259522560(253440K) max = 259522560(253440K)
 06/08/2010 10.47.47:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
 LoadingFiles:WARNING: Memory low after Garbage Collect:init = 16777216(16384K) u
 sed = 246893272(241106K) committed = 259522560(253440K) max = 259522560(253440K)
 
 06/08/2010 10.47.47:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
 LoadingFiles:SEVERE: Memory too low:init = 16777216(16384K) used = 226006856(220
 709K) committed = 259522560(253440K) max = 259522560(253440K)
 


EDIT: Or is it Windows' prefetch and standby memory being retarded?
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7487
Offline

Your memory settings dont seem to have taken effect, are you editing jaikoz.bat but then running Jaikoz.exe, you need to run Jaikoz.bat

thanks Paul (Administrator)
MAzE5h1p69wB

Pro

Joined: 21/04/2010 14:05:13
Messages: 83
Offline

Could you point out to me what is wrong in this jaikoz.bat that I am using?
Code:
 @echo off
  set CLASSPATH=
  java -jar lib\jaikoz.jar -l2 -m2 -f -Xms150m -Xmx3200m -XX:MaxPermSize=150m
 


And yes these settings do take effect.

Also it seems that jaikoz / java is using the "Free" memory, then requests more which released "Standby" memory that superfetch fills with stupidity geared for dumb applications like office, uses a lot of cpu in posting these requests each for a minuscule amount of ram such as "gimme 4 MiB more" and stops after a set number of times.

EDIT: Oh and then it craps all over itself because it runs out of perm memory and helpfully informs that maybe the VM needs to be taken out back and shot in the head.
Code:
 
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
 bstractAudioFileBridge.java:121)
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
 FileBridge.java:60)
 06/08/2010 11.27.47:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Uncaught throwableermGen space on thread:AudioFileLoader_Worker_7 caught by
 JaikozThreadGroup
 06/08/2010 11.28.19:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Terminating application because a serious error has occurred
 java.lang.OutOfMemoryError: PermGen space
         at java.lang.Class.getDeclaredMethods0(Native Method)
         at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
         at java.lang.Class.getDeclaredMethod(Unknown Source)
         at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
         at java.io.ObjectStreamClass.access$1700(Unknown Source)
         at java.io.ObjectStreamClass$2.run(Unknown Source)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.io.ObjectStreamClass.<init>(Unknown Source)
         at java.io.ObjectStreamClass.lookup(Unknown Source)
         at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
         at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
         at java.io.ObjectInputStream.readClassDesc(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at java.util.HashMap.readObject(Unknown Source)
         at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
         at java.lang.reflect.Method.invoke(Unknown Source)
         at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
         at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)
 
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
 bstractAudioFileBridge.java:121)
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
 FileBridge.java:60)
 06/08/2010 11.27.41:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Uncaught throwableermGen space on thread:AudioFileLoader_Worker_4 caught by
 JaikozThreadGroup
 06/08/2010 11.28.19:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Uncaught throwableermGen space on thread:AudioFileLoader_Worker_1 caught by
 JaikozThreadGroup
 06/08/2010 11.28.20:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Terminating application because a serious error has occurred
 java.lang.OutOfMemoryError: PermGen space
         at java.lang.Class.getDeclaredMethods0(Native Method)
         at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
         at java.lang.Class.getDeclaredMethod(Unknown Source)
         at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
         at java.io.ObjectStreamClass.access$1700(Unknown Source)
         at java.io.ObjectStreamClass$2.run(Unknown Source)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.io.ObjectStreamClass.<init>(Unknown Source)
         at java.io.ObjectStreamClass.lookup(Unknown Source)
         at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
         at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
         at java.io.ObjectInputStream.readClassDesc(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at java.util.HashMap.readObject(Unknown Source)
         at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
         at java.lang.reflect.Method.invoke(Unknown Source)
         at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
         at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)
 
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
 bstractAudioFileBridge.java:121)
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
 FileBridge.java:60)
 06/08/2010 11.28.16:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Uncaught throwableermGen space on thread:AudioFileLoader_Worker_2 caught by
 JaikozThreadGroup
 06/08/2010 11.28.16:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Uncaught throwableermGen space on thread:AudioFileLoader_Worker_5 caught by
 JaikozThreadGroup
 06/08/2010 11.27.52:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Terminating application because a serious error has occurred
 java.lang.OutOfMemoryError: PermGen space
         at java.lang.Class.getDeclaredMethods0(Native Method)
         at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
         at java.lang.Class.getDeclaredMethod(Unknown Source)
         at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
         at java.io.ObjectStreamClass.access$1700(Unknown Source)
         at java.io.ObjectStreamClass$2.run(Unknown Source)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.io.ObjectStreamClass.<init>(Unknown Source)
         at java.io.ObjectStreamClass.lookup(Unknown Source)
         at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
         at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
         at java.io.ObjectInputStream.readClassDesc(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at java.util.HashMap.readObject(Unknown Source)
         at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
         at java.lang.reflect.Method.invoke(Unknown Source)
         at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
         at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)
 
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
 bstractAudioFileBridge.java:121)
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
 FileBridge.java:60)
 06/08/2010 11.27.52:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Uncaught throwableermGen space on thread:AudioFileLoader_Worker_6 caught by
 JaikozThreadGroup
 06/08/2010 11.28.22:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Terminating application because a serious error has occurred
 java.lang.OutOfMemoryError: PermGen space
 06/08/2010 11.28.22:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Terminating application because a serious error has occurred
 java.lang.OutOfMemoryError: PermGen space
         at java.lang.ClassLoader.defineClass1(Native Method)
         at java.lang.ClassLoader.defineClassCond(Unknown Source)
         at java.lang.ClassLoader.defineClass(Unknown Source)
         at java.security.SecureClassLoader.defineClass(Unknown Source)
         at java.net.URLClassLoader.defineClass(Unknown Source)
         at java.net.URLClassLoader.access$000(Unknown Source)
         at java.net.URLClassLoader$1.run(Unknown Source)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(Unknown Source)
         at java.lang.ClassLoader.loadClass(Unknown Source)
         at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
         at java.lang.ClassLoader.loadClass(Unknown Source)
         at com.mchange.v2.c3p0.impl.NewPooledConnection.handleThrowable(NewPoole
 dConnection.java:432)
         at com.mchange.v2.c3p0.impl.NewProxyStatement.executeQuery(NewProxyState
 ment.java:52)
         at com.jthink.jaikoz.db.Db.runQuery(Db.java:412)
         at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:69)
 
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
 bstractAudioFileBridge.java:121)
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
 FileBridge.java:60)
         at com.jthink.jaikoz.data.AudioFileFromDirectoryAndFilesLoader$TaskWorke
 rThread.run(AudioFileFromDirectoryAndFilesLoader.java:660)
 06/08/2010 11.28.23:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Terminating application because a serious error has occurred
 java.lang.OutOfMemoryError: PermGen space
         at java.lang.Class.getDeclaredMethods0(Native Method)
         at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
         at java.lang.Class.getDeclaredMethod(Unknown Source)
         at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
         at java.io.ObjectStreamClass.access$1700(Unknown Source)
         at java.io.ObjectStreamClass$2.run(Unknown Source)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.io.ObjectStreamClass.<init>(Unknown Source)
         at java.io.ObjectStreamClass.lookup(Unknown Source)
         at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
         at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
         at java.io.ObjectInputStream.readClassDesc(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at java.util.HashMap.readObject(Unknown Source)
         at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
         at java.lang.reflect.Method.invoke(Unknown Source)
         at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
         at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)
 
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
 bstractAudioFileBridge.java:121)
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
 FileBridge.java:60)
 06/08/2010 11.28.35:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Terminating application because a serious error has occurred
 java.lang.OutOfMemoryError: PermGen space
         at java.lang.Class.getDeclaredMethods0(Native Method)
         at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
         at java.lang.Class.getDeclaredMethod(Unknown Source)
         at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
         at java.io.ObjectStreamClass.access$1700(Unknown Source)
         at java.io.ObjectStreamClass$2.run(Unknown Source)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.io.ObjectStreamClass.<init>(Unknown Source)
         at java.io.ObjectStreamClass.lookup(Unknown Source)
         at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
         at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
         at java.io.ObjectInputStream.readClassDesc(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at java.util.HashMap.readObject(Unknown Source)
         at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
         at java.lang.reflect.Method.invoke(Unknown Source)
         at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
         at java.io.ObjectInputStream.readSerialData(Unknown Source)
         at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
         at java.io.ObjectInputStream.readObject0(Unknown Source)
         at java.io.ObjectInputStream.readObject(Unknown Source)
         at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
         at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)
 
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
 bstractAudioFileBridge.java:121)
         at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
 FileBridge.java:60)
 06/08/2010 11.28.35:com.jthink.jaikoz.manipulate.RunAndWaitForGUIUpdate:updateGU
 I:SEVERE: RunAndWaitForGUIUpdate:invocationExceptionPermGen space
 06/08/2010 11.28.50:com.jthink.jaikoz.manipulate.RunAndWaitForGUIUpdate:updateGU
 I:SEVERE: RunAndWaitForGUIUpdate:invocationExceptionPermGen space
 06/08/2010 11.28.45:com.jthink.jaikoz.manipulate.RunAndWaitForGUIUpdate:updateGU
 I:SEVERE: RunAndWaitForGUIUpdate:invocationExceptionPermGen space
 Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
 occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
 erminated
 Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
 occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
 erminated
 Java HotSpot(TM) 64-Bit Server VM warning: 06/08/2010 11.29.13:com.jthink.jaikoz
 .JaikozThreadGroup:uncaughtException:SEVERE: Error in JaikozThreadGroup.uncaught
 Exception(),continue termination
 java.lang.OutOfMemoryError: PermGen space
 06/08/2010 11.29.34:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
 : Error in JaikozThreadGroup.uncaughtException(),continue termination
 java.lang.OutOfMemoryError: PermGen space
 Exception java.lang.OutOfMemoryError occurred dispatching signal UNKNOWN to hand
 ler- the VM may need to be forcibly terminated
 Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
 occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
 erminated
 Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
 occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
 erminated
 Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
 occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
 erminated
 
MAzE5h1p69wB

Pro

Joined: 21/04/2010 14:05:13
Messages: 83
Offline

Well I increased the perm store thing which java uses because it can't relate to hardware in this case memory:

JAIKOZ.BAT
Code:
 java -jar lib\jaikoz.jar -l2 -m2 -f -Xms150m -Xmx3200m -XX:MaxPermSize=2000m
 


But it still won't work even though there is a gig of standby memory and 200 mib of free memory right there waiting for it.

Anything I can do to help debug further?

As a suggestion though, I would recommend that possibly writing an application that can use memory would be a good idea.

EDIT: Oh and it barfs at around 20000 songs, and disabling show artwork does not help of course.
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7487
Offline

In your initial log it says
Code:
 max = 259522560(253440K)
 

i.e 253 MB, which doesnt match up with your jaikoz.bat file, so thats why Im asking are you actually running jaikoz.bat rather than jaikoz.exe.

But aside from that if it is continually barfing at one point despite increasing memory then it is likely it is falling over on one specific file because of a problem decoding the image, not displaying the image column does not have any effect the image is still decoded.

I have put in workarounds for this, but maybe there is a another case that needs fixing. Run your load a few times and if it continually fails at one point try removing the subfolder where it is failing, then try loading just that folder and see if you can break it consistently and send me some details.

The Permgen problem is a deficiency in Java that I am going to have work around, I use String.intern() to ensure only one copy of strings found in the metadata is stored in memory thus reducing the total memory used. However unfortunately it doesnt store this in Heap but in Permanent and its difficult to assess how much Perm may be needed, I may remoce use of intern().

thanks Paul (Administrator)
MAzE5h1p69wB

Pro

Joined: 21/04/2010 14:05:13
Messages: 83
Offline

All of these files have previously been run through Jaikoz, and been processed with the setting delete id3v1 always write v2 in utf-8.

At this point I am quite frustrated, given the issues with disk-queue, memory, not saving tracks with correct filenames, and so on.

It would be good if you could fetch a few tens of thousands of music files from the internet, and set yourself the goal of making it possible to run Jaikoz against more than a few thousand files, no matter what tiny little errors are present in them or what happens.
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7487
Offline

MAzE5h1p69wB wrote:
All of these files have previously been run through Jaikoz, and been processed with the setting delete id3v1 always write v2 in utf-8.
 

Dont understand the relevance

MAzE5h1p69wB wrote:

At this point I am quite frustrated, given the issues with disk-queue, memory, not saving tracks with correct filenames, and so on.
 

Jaikoz is complex software, yes its not perfect but ive been working to resolve your issues and have already provided you with fixes for some of the problems you mention.

MAzE5h1p69wB wrote:

It would be good if you could fetch a few tens of thousands of music files from the internet, and set yourself the goal of making it possible to run Jaikoz against more than a few thousand files, no matter what tiny little errors are present in them or what happens.
 

I think if you limit yourself 10,000 songs you should be okay, and that is going to save you lot of time over anhy other method, also if you could resend your support files it will give me some clues about whther you are hitting bad image problem.



thanks Paul (Administrator)
 
Forum Index -> Jaikoz Issues
Go to:   
Powered by JForum 2.1.6 © JForum Team