Home » Forum Home » Development

Topic: Server woes - JVM crashes
Replies: 3   Views: 28,097   Pages: 1   Last Post: May 24, 2005, 3:47 PM by: dweebo

Search Forum

Back to Topic List Topics: [ Previous | Next ]
Replies: 3   Views: 28,097   Pages: 1  
dweebo

Posts: 1,032
Registered: Dec 16, 2001
From: Powell, OH
Age: 37
Home page
Server woes - JVM crashes
Posted: Mar 10, 2005, 3:19 PM

So you might have noticed the server still crashes on occasion. There are currently 2 problems I know of that have been happening since the server move.

The first on is a java bug, so not a whole lot I can do about it. It seems to occur once a week or so. The JVM crashes when trying to resolve the localhost name, but randomly. Here is the JVM message, maybe someone will find this via google and have an answer, I couldn't find any solutions anywere. From similar bug reports at Sun, it seems like most likely the native call to get the local host name returns an answer the JVM doesn't know how to handle and then everything blows up. I can't stop calling that function, it is in tomcat and in jive (forum software).


An unexpected exception has been detected in native code outside the VM.
Unexpected Signal : 11 occurred at PC=0x40088346
Function=fgets_unlocked+0x26
Library=/lib/libc.so.6

Current Java thread:
at java.net.Inet4AddressImpl.getHostByAddr(Native Method)
at java.net.InetAddress$1.getHostByAddr(InetAddress.java:774)
at java.net.InetAddress.getHostFromNameService(InetAddress.java:464)
at java.net.InetAddress.getHostName(InetAddress.java:407)
at java.net.InetAddress.getHostName(InetAddress.java:379)
at org.apache.coyote.http11.Http11Processor.parseHost(Http11Processor.java:1280)
at org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:1248)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:783)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:534)

Dynamic libraries:
08048000-0804f000 r-xp 00000000 03:03 8437792 /usr/local/jakarta-tomcat-5.0.28/bin/jsvc
0804f000-08050000 rw-p 00006000 03:03 8437792 /usr/local/jakarta-tomcat-5.0.28/bin/jsvc
40000000-40016000 r-xp 00000000 03:03 163853 /lib/ld-2.3.2.so.dpkg-new (deleted)
40016000-40017000 rw-p 00015000 03:03 163853 /lib/ld-2.3.2.so.dpkg-new (deleted)
4001a000-4001b000 r--s 00000000 03:03 4030507 /usr/local/jakarta-tomcat-5.0.28/common/lib/naming-java.jar
4001b000-4001c000 r--s 00000000 03:03 4161587 /usr/local/jakarta-tomcat-5.0.28/server/lib/jkshm.jar
4001c000-4001e000 r-xp 00000000 03:03 163931 /lib/libdl-2.3.2.so.dpkg-new (deleted)
4001e000-4001f000 rw-p 00002000 03:03 163931 /lib/libdl-2.3.2.so.dpkg-new (deleted)
4001f000-40147000 r-xp 00000000 03:03 163929 /lib/libc-2.3.2.so.dpkg-new (deleted)
40147000-4014f000 rw-p 00127000 03:03 163929 /lib/libc-2.3.2.so.dpkg-new (deleted)
40153000-4015b000 r-xp 00000000 03:03 163938 /lib/libnss_files-2.3.2.so.dpkg-new (deleted)
4015b000-4015c000 rw-p 00008000 03:03 163938 /lib/libnss_files-2.3.2.so.dpkg-new (deleted)
4015c000-40558000 r-xp 00000000 03:03 7061528 /usr/local/j2sdk1.4.2_06/jre/lib/i386/client/libjvm.so
40558000-40574000 rw-p 003fb000 03:03 7061528 /usr/local/j2sdk1.4.2_06/jre/lib/i386/client/libjvm.so
40586000-4058a000 rw-s 00000000 03:03 7110681 /tmp/hsperfdata_tomcat/3703
4058a000-4059c000 r-xp 00000000 03:03 163934 /lib/libnsl-2.3.2.so.dpkg-new (deleted)
4059c000-4059d000 rw-p 00011000 03:03 163934 /lib/libnsl-2.3.2.so.dpkg-new (deleted)
4059f000-405c0000 r-xp 00000000 03:03 163932 /lib/libm-2.3.2.so.dpkg-new (deleted)
405c0000-405c1000 rw-p 00020000 03:03 163932 /lib/libm-2.3.2.so.dpkg-new (deleted)
405c1000-405ce000 r-xp 00000000 03:03 163943 /lib/libpthread-0.10.so.dpkg-new (deleted)
405ce000-405d0000 rw-p 0000c000 03:03 163943 /lib/libpthread-0.10.so.dpkg-new (deleted)
40612000-4061a000 r-xp 00000000 03:03 6733845 /usr/local/j2sdk1.4.2_06/jre/lib/i386/native_threads/libhpi.so
4061a000-4061b000 rw-p 00007000 03:03 6733845 /usr/local/j2sdk1.4.2_06/jre/lib/i386/native_threads/libhpi.so
4061b000-4062b000 r-xp 00000000 03:03 6701098 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libverify.so
4062b000-4062d000 rw-p 0000f000 03:03 6701098 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libverify.so
4062d000-4064d000 r-xp 00000000 03:03 6701099 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libjava.so
4064d000-4064f000 rw-p 0001f000 03:03 6701099 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libjava.so
4064f000-40663000 r-xp 00000000 03:03 6701101 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libzip.so
40663000-40666000 rw-p 00013000 03:03 6701101 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libzip.so
40666000-4075d000 r--s 00000000 03:03 2981901 /usr/local/jakarta-tomcat-5.0.28/common/endorsed/xercesImpl.jar
4075d000-4077c000 r--s 00000000 03:03 2981902 /usr/local/jakarta-tomcat-5.0.28/common/endorsed/xml-apis.jar
4077c000-42123000 r--s 00000000 03:03 6176855 /usr/local/j2sdk1.4.2_06/jre/lib/rt.jar
4216d000-42183000 r--s 00000000 03:03 6176807 /usr/local/j2sdk1.4.2_06/jre/lib/sunrsasign.jar
42183000-42260000 r--s 00000000 03:03 6176852 /usr/local/j2sdk1.4.2_06/jre/lib/jsse.jar
42260000-42271000 r--s 00000000 03:03 6176808 /usr/local/j2sdk1.4.2_06/jre/lib/jce.jar
42271000-427ca000 r--s 00000000 03:03 6176853 /usr/local/j2sdk1.4.2_06/jre/lib/charsets.jar
44872000-44875000 r--s 00000000 03:03 8093723 /usr/local/j2sdk1.4.2_06/jre/lib/ext/dnsns.jar
44875000-44878000 r--s 00000000 03:03 1720360 /usr/local/jakarta-tomcat-5.0.28/bin/commons-daemon.jar
44878000-44880000 r--s 00000000 03:03 1720355 /usr/local/jakarta-tomcat-5.0.28/bin/bootstrap.jar
5e813000-5e82f000 r--s 00000000 03:03 8093722 /usr/local/j2sdk1.4.2_06/jre/lib/ext/sunjce_provider.jar
5e82f000-5e83c000 r--s 00000000 03:03 8093725 /usr/local/j2sdk1.4.2_06/jre/lib/ext/ldapsec.jar
5e83c000-5e8f8000 r--s 00000000 03:03 8093726 /usr/local/j2sdk1.4.2_06/jre/lib/ext/localedata.jar
5e8f8000-5edd4000 r--s 00000000 03:03 360480 /usr/local/j2sdk1.4.2_06/lib/tools.jar
5edd4000-5ee28000 r--s 00000000 03:03 1720365 /usr/local/jakarta-tomcat-5.0.28/bin/jmx.jar
5ee28000-5ee2f000 r--s 00000000 03:03 1720362 /usr/local/jakarta-tomcat-5.0.28/bin/commons-logging-api.jar
5ee2f000-5ee3f000 r-xp 00000000 03:03 6701104 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libnet.so
5ee3f000-5ee40000 rw-p 0000f000 03:03 6701104 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libnet.so
5ee40000-5ee43000 r--s 00000000 03:03 4030496 /usr/local/jakarta-tomcat-5.0.28/common/lib/ant-launcher.jar
5ee43000-5ef2e000 r--s 00000000 03:03 4030497 /usr/local/jakarta-tomcat-5.0.28/common/lib/ant.jar
5ef2e000-5efb7000 r--s 00000000 03:03 4030498 /usr/local/jakarta-tomcat-5.0.28/common/lib/commons-collections-3.1.jar
5efb7000-5efd2000 r--s 00000000 03:03 4030499 /usr/local/jakarta-tomcat-5.0.28/common/lib/commons-dbcp-1.2.1.jar
5efd2000-5efee000 r--s 00000000 03:03 4030500 /usr/local/jakarta-tomcat-5.0.28/common/lib/commons-el.jar
5efee000-5eff9000 r--s 00000000 03:03 4030501 /usr/local/jakarta-tomcat-5.0.28/common/lib/commons-pool-1.2.jar
5eff9000-5f04f000 r--s 00000000 03:03 4030502 /usr/local/jakarta-tomcat-5.0.28/common/lib/jasper-compiler.jar
5f04f000-5f069000 r--s 00000000 03:03 4030503 /usr/local/jakarta-tomcat-5.0.28/common/lib/jasper-runtime.jar
5f069000-5f076000 r--s 00000000 03:03 4030504 /usr/local/jakarta-tomcat-5.0.28/common/lib/jsp-api.jar
5f076000-5f07d000 r--s 00000000 03:03 4030505 /usr/local/jakarta-tomcat-5.0.28/common/lib/naming-common.jar
5f07d000-5f081000 r--s 00000000 03:03 4030506 /usr/local/jakarta-tomcat-5.0.28/common/lib/naming-factory.jar
5f081000-5f08c000 r--s 00000000 03:03 4030508 /usr/local/jakarta-tomcat-5.0.28/common/lib/naming-resources.jar
5f08c000-5f0a4000 r--s 00000000 03:03 4030509 /usr/local/jakarta-tomcat-5.0.28/common/lib/servlet-api.jar
5f0a4000-5f0de000 r--s 00000000 03:03 4030493 /usr/local/jakarta-tomcat-5.0.28/common/lib/mysql-connector-java-3.0.16-ga-bin.jar
5f0de000-5f0e3000 r--s 00000000 03:03 4161574 /usr/local/jakarta-tomcat-5.0.28/server/lib/catalina-ant.jar
5f0e3000-5f102000 r--s 00000000 03:03 4161575 /usr/local/jakarta-tomcat-5.0.28/server/lib/catalina-cluster.jar
5f102000-5f108000 r--s 00000000 03:03 4161576 /usr/local/jakarta-tomcat-5.0.28/server/lib/catalina-i18n-es.jar
5f108000-5f10e000 r--s 00000000 03:03 4161577 /usr/local/jakarta-tomcat-5.0.28/server/lib/catalina-i18n-fr.jar
5f10e000-5f114000 r--s 00000000 03:03 4161578 /usr/local/jakarta-tomcat-5.0.28/server/lib/catalina-i18n-ja.jar
5f114000-5f12f000 r--s 00000000 03:03 4161579 /usr/local/jakarta-tomcat-5.0.28/server/lib/catalina-optional.jar
5f12f000-5f1dc000 r--s 00000000 03:03 4161580 /usr/local/jakarta-tomcat-5.0.28/server/lib/catalina.jar
5f1dc000-5f20b000 r--s 00000000 03:03 4161581 /usr/local/jakarta-tomcat-5.0.28/server/lib/commons-beanutils.jar
5f20b000-5f226000 r--s 00000000 03:03 4161582 /usr/local/jakarta-tomcat-5.0.28/server/lib/commons-digester.jar
5f226000-5f22c000 r--s 00000000 03:03 4161583 /usr/local/jakarta-tomcat-5.0.28/server/lib/commons-fileupload-1.0.jar
5f22c000-5f247000 r--s 00000000 03:03 4161584 /usr/local/jakarta-tomcat-5.0.28/server/lib/commons-modeler.jar
5f247000-5f24e000 r--s 00000000 03:03 4161585 /usr/local/jakarta-tomcat-5.0.28/server/lib/jakarta-regexp-1.3.jar
5f24e000-5f256000 r--s 00000000 03:03 4161586 /usr/local/jakarta-tomcat-5.0.28/server/lib/jkconfig.jar
5f256000-5f257000 r--s 00000000 03:03 4161589 /usr/local/jakarta-tomcat-5.0.28/server/lib/servlets-common.jar
5f257000-5f25d000 r--s 00000000 03:03 4161590 /usr/local/jakarta-tomcat-5.0.28/server/lib/servlets-default.jar
5f25d000-5f25f000 r--s 00000000 03:03 4161591 /usr/local/jakarta-tomcat-5.0.28/server/lib/servlets-invoker.jar
5f25f000-5f265000 r--s 00000000 03:03 4161593 /usr/local/jakarta-tomcat-5.0.28/server/lib/servlets-webdav.jar
5f265000-5f26a000 r--s 00000000 03:03 4161594 /usr/local/jakarta-tomcat-5.0.28/server/lib/tomcat-coyote.jar
5f26a000-5f277000 r--s 00000000 03:03 4161595 /usr/local/jakarta-tomcat-5.0.28/server/lib/tomcat-http11.jar
5f277000-5f278000 r--s 00000000 03:03 4161596 /usr/local/jakarta-tomcat-5.0.28/server/lib/tomcat-jk.jar
5f278000-5f296000 r--s 00000000 03:03 4161597 /usr/local/jakarta-tomcat-5.0.28/server/lib/tomcat-jk2.jar
5f296000-5f29a000 r--s 00000000 03:03 4161598 /usr/local/jakarta-tomcat-5.0.28/server/lib/tomcat-jni.jar
5f29a000-5f2c6000 r--s 00000000 03:03 4161599 /usr/local/jakarta-tomcat-5.0.28/server/lib/tomcat-util.jar
5f2c6000-5f319000 r-xp 00000000 03:03 6701108 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libmlib_image.so
5f319000-5f31a000 rw-p 00052000 03:03 6701108 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libmlib_image.so
5f31a000-5f321000 r-xp 00000000 03:03 5095449 /usr/X11R6/lib/libXp.so.6.2
5f321000-5f322000 rw-p 00006000 03:03 5095449 /usr/X11R6/lib/libXp.so.6.2
5f322000-5f370000 r-xp 00000000 03:03 5095451 /usr/X11R6/lib/libXt.so.6.0
5f370000-5f374000 rw-p 0004d000 03:03 5095451 /usr/X11R6/lib/libXt.so.6.0
5f374000-5f381000 r-xp 00000000 03:03 5095447 /usr/X11R6/lib/libXext.so.6.4
5f381000-5f382000 rw-p 0000c000 03:03 5095447 /usr/X11R6/lib/libXext.so.6.4
5f382000-5f386000 r-xp 00000000 03:03 5095453 /usr/X11R6/lib/libXtst.so.6.1
5f386000-5f387000 rw-p 00004000 03:03 5095453 /usr/X11R6/lib/libXtst.so.6.1
5f387000-5f463000 r-xp 00000000 03:03 5095445 /usr/X11R6/lib/libX11.so.6.2
5f463000-5f466000 rw-p 000db000 03:03 5095445 /usr/X11R6/lib/libX11.so.6.2
5f466000-5f46e000 r-xp 00000000 03:03 5095443 /usr/X11R6/lib/libSM.so.6.0
5f46e000-5f46f000 rw-p 00007000 03:03 5095443 /usr/X11R6/lib/libSM.so.6.0
5f46f000-5f483000 r-xp 00000000 03:03 5095441 /usr/X11R6/lib/libICE.so.6.3
5f483000-5f484000 rw-p 00013000 03:03 5095441 /usr/X11R6/lib/libICE.so.6.3
5f486000-5f4bd000 r-xp 00000000 03:03 6701113 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libjpeg.so
5f4bd000-5f4be000 rw-p 00036000 03:03 6701113 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libjpeg.so
5f4d9000-5f719000 r--s 00000000 03:03 983067 /dsg/dev/webapp/WEB-INF/lib/jivebase.jar
5f719000-5f7d3000 r-xp 00000000 03:03 6701112 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libfontmanager.so
5f7d3000-5f7ee000 rw-p 000b9000 03:03 6701112 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libfontmanager.so
5f852000-5f856000 r--s 00000000 03:03 983065 /dsg/dev/webapp/WEB-INF/lib/pop3.jar
5f85a000-5f85d000 r-xp 00000000 03:03 163936 /lib/libnss_dns-2.3.2.so.dpkg-new (deleted)
5f85d000-5f85e000 rw-p 00003000 03:03 163936 /lib/libnss_dns-2.3.2.so.dpkg-new (deleted)
5f85e000-5f86d000 r-xp 00000000 03:03 163944 /lib/libresolv-2.3.2.so.dpkg-new (deleted)
5f86d000-5f86e000 rw-p 0000f000 03:03 163944 /lib/libresolv-2.3.2.so.dpkg-new (deleted)
5fa00000-5fcd1000 r-xp 00000000 03:03 6701109 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libawt.so
5fcd1000-5fce7000 rw-p 002d0000 03:03 6701109 /usr/local/j2sdk1.4.2_06/jre/lib/i386/libawt.so

Heap at VM Abort:
Heap
def new generation total 24768K, used 20688K [0x44880000, 0x46360000, 0x46360000)
eden space 22016K, 92% used [0x44880000, 0x45c69748, 0x45e00000)
from space 2752K, 10% used [0x460b0000, 0x460fa960, 0x46360000)
to space 2752K, 0% used [0x45e00000, 0x45e00000, 0x460b0000)
tenured generation total 330880K, used 227882K [0x46360000, 0x5a680000, 0x5a680000)
the space 330880K, 68% used [0x46360000, 0x541eaba8, 0x541eac00, 0x5a680000)
compacting perm gen total 24064K, used 23988K [0x5a680000, 0x5be00000, 0x5e680000)
the space 24064K, 99% used [0x5a680000, 0x5bded090, 0x5bded200, 0x5be00000)

Local Time = Thu Jan 13 17:28:25 2005
Elapsed Time = 278970
#
# The exception above was detected in native code outside the VM
#
# Java VM: Java HotSpot(TM) Client VM (1.4.2_06-b03 mixed mode)
#

I'm running Debian woody, with some packages from sarge (including libc6, the latest version 2.3.2.ds1-20).

The localhost should be determined by my DNS server running on the same machine according to my /etc/nsswitch.conf file:
...
hosts: files dns
...

/etc/hosts doesn't list the IP for pente.org so it shouldn't find it there and then should try dns.

My dns configuration isn't anything fancy, and I think everything is ok for pente.org.

I don't know, this is an annoying one.

The second problem I just noticed twice in the past week, the server doesn't crash, it hangs. This is probably a deadlock caused by improper synchronization somewhere in my code. It might have been caused by some of the recent changes, I'm not sure. I will have to review what I've added recently and see if I can find it. Recently I have been adding alot of code that connects the game server to code serving html pages, that always worries me a little since it opens up many possibilities for these types of synchronization problems.

Anyways, hang in there, I'm working on resolving the server problems!

Pente Rocks!

dweebo

Posts: 1,032
Registered: Dec 16, 2001
From: Powell, OH
Age: 37
Home page
Re: Server woes - JVM crashes
Posted: Apr 21, 2005, 3:31 PM

I've upgraded the server to use JDK1.5.0_02 to see if that helps, plus I want to start using some of the new Java 5 features.

I also upgraded to Tomcat 5.5.9. We'll see what happens.

Pente Rocks!
dweebo

Posts: 1,032
Registered: Dec 16, 2001
From: Powell, OH
Age: 37
Home page
Re: Server woes - JVM crashes
Posted: Apr 30, 2005, 8:21 PM

Nope, JDK1.5 is not the solution, sigh. Has crashed twice since the upgrade.

Just upgraded libc6 though, which is where the problem is, so maybe that will do it....

Pente Rocks!
dweebo

Posts: 1,032
Registered: Dec 16, 2001
From: Powell, OH
Age: 37
Home page
Re: Server woes - JVM crashes
Posted: May 24, 2005, 3:47 PM

Nope, still happening.

Here is my latest attempt. It happens doing reverse DNS lookups (not all the time, just what seems to me to be randomly). My server resolves DNS lookups by first using the nameserver on the same machine (I run my own DNS server for pente.org). However, I don't provide reverse DNS entries, since my hosting company has the authority for that, so they host my reverse DNS entry.

So, I changed the server to not even look at my local nameserver, instead to jump right to my hosting company's nameserver. I don't know, maybe that will do the trick.

If not, I plan on modifying either the underlying code of the JVM (not sure about that), or I will modify the underlying linux code, my modifications would just hardcode in the IP of pente.org so as to always handle it correctly.

Pente Rocks!
Replies: 3   Views: 28,097   Pages: 1  
Back to Topic List
Topics: [ Previous | Next ]


Powered by Jive Software