debugging - jps returns no output even when java processes are running

Question

I'm trying to debug some issues with java processes on a Solaris box, but running jps returns no output. And jstack gives the error 'Permission denied'. The box is part of a cluster of 3 identical servers, jps and jstack work fine on the 2 other servers.

I found the following forum post from someone with the same problem but no answers: http://forums.sun.com/thread.jspa?threadID=5422237

To clarify running bps and grep for java gives all java proceses correctly, but jps gives nothing (anonymised with 'program' and 'client' to protect the guilty):

program @ clientdelivery2 : ~/
-> bps auxww|grep java
program     3427  5.5 54.067742726649544 ?        S   Sep 25 1039:47 /usr/jdk/instances/jdk1.6.0_16/bin/amd64/java -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/app/client/program/tomcat/conf/logging.properties -Xmx6144m -XX:PermSize=128m -XX:MaxPermSize=512m -Djava.endorsed.dirs=/app/client/program/tomcat/endorsed -classpath :/app/client/program/tomcat/bin/bootstrap.jar -Dcatalina.base=/app/client/program/tomcat -Dcatalina.home=/app/client/program/tomcat -Djava.io.tmpdir=/app/client/program/tomcat/temp org.apache.catalina.startup.Bootstrap start
program    29915  0.1 11.915252441467896 ?        S 14:55:28  3:59 /usr/jdk/instances/jdk1.6.0_16/bin/amd64/java -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/app/clientclone/program/tomcat/conf/logging.properties -Xmx2g -XX:PermSize=128m -XX:MaxPermSize=512m -Dcom.sun.management.jmxremote -Djava.endorsed.dirs=/app/clientclone/program/tomcat/endorsed -classpath :/app/clientclone/program/tomcat/bin/bootstrap.jar -Dcatalina.base=/app/clientclone/program/tomcat -Dcatalina.home=/app/clientclone/program/tomcat -Djava.io.tmpdir=/app/clientclone/program/tomcat/temp org.apache.catalina.startup.Bootstrap start
program     1573  0.0  0.0 4760 1332 pts/5    S 17:05:24  0:00 grep --colour java

program @ clientdelivery2 : ~/
-> jps

program @ clientdelivery2 : ~/
->

I asked around and from here http://forums.oracle.com/forums/message.jspa?messageID=5408592 I've got that the issue is:

12460/2: mkdir("/tmp/hsperfdata_program", 0755) Err#13 EACCES [ALL]

Meaning jps is being denied access to the psperfdata directory.

Has anyone run into this issue and know how to solve it?

Answer 1

Turns out the user didn't have access to /tmp because of some issue with mounting the file system. This leads to the files inside hsperfdata_ to never be written, even though the user had access to the /tmp/hsperfdata_ folder itself.