SoftTree Technologies SoftTree Technologies
Technical Support Forums
RegisterSearchFAQMemberlistUsergroupsLog in
java.lang.OutOfMemoryError: PermGen space

 
Reply to topic    SoftTree Technologies Forum Index » 24x7 Scheduler, Event Server, Automation Suite View previous topic
View next topic
java.lang.OutOfMemoryError: PermGen space
Author Message
Dominik



Joined: 30 Jul 2007
Posts: 36

Post java.lang.OutOfMemoryError: PermGen space Reply with quote
Dear Support Team,

I've got the following entry/error in debug.log:


2014-08-06 10:03:00,155 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): starting JavaScript
2014-08-06 10:03:00,155 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): job not using user authentication
2014-08-06 10:03:00,155 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - evaluateScript(): starting jsthread
2014-08-06 10:03:00,155 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - evaluateScript(): waiting
2014-08-06 10:03:00,155 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - jsthread started
2014-08-06 10:03:00,155 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptRunner - runScript: JavaScriptRunner
2014-08-06 10:03:00,155 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptRunner - Connecting to database...
2014-08-06 10:03:00,155 [JavaScript] DEBUG com.softtreetech.jscheduler.business.sql.SqlConnector - loadDriver() driverPath=./drivers/ora/ojdbc14_g.jar driverClass=oracle.jdbc.OracleDriver
2014-08-06 10:03:00,155 [JavaScript] DEBUG com.softtreetech.jscheduler.business.sql.SqlConnector - parsed urls: 1
2014-08-06 10:03:00,155 [JavaScript] DEBUG com.softtreetech.jscheduler.business.sql.SqlConnector - url #0 file:/C:/Program Files/24x7/drivers/ora/ojdbc14_g.jar
2014-08-06 10:03:00,155 [JavaScript] DEBUG com.softtreetech.jscheduler.business.sql.SqlConnector - java.net.URLClassLoader@1846044
2014-08-06 10:03:00,420 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - java.lang.OutOfMemoryError: PermGen space
2014-08-06 10:03:00,420 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - jsthread stopped
2014-08-06 10:03:00,420 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - evaluateScript(): script terminated with exception: java.lang.OutOfMemoryError: PermGen space
2014-08-06 10:03:00,420 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): script evaluated
2014-08-06 10:03:00,420 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): stopping local js thread
2014-08-06 10:03:00,420 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - stopJScript()
2014-08-06 10:03:00,420 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): job finished
2014-08-06 10:03:00,420 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.queue.JobQueue - QUEUE_UNLOCKED


running the following (schedule polling 1min) script:
( i'm looking in a table to get schedulerId entered by other programs and want to execute them)
This works fine, but after a while the scheduler hangs up, i cant close the tool anymore and have to kill it (Java (TM) Platform SE Binary) via taskmanager)


var job_id=0;
var internal_id=0;
var ret_val_str="";
var sql="";

Database.connect("DWH");


sql = "select ref.TAB_KEY from STG01.Z_DWH_SCHEDULER_STEUERUNG s, ( select min(SOURCE_SERVER || '|' || SOURCE_JOBID || '|' || TARGET_SERVER || '|' || TARGET_JOBID || '|' || to_char(JOB_INSERT_TIME,'yyyymmddHH24MISS')) as TAB_KEY from STG01.Z_DWH_SCHEDULER_STEUERUNG where TARGET_SERVER = 'bdopis1' and JOB_STATUS = 1 ) ref where ref.TAB_KEY = (SOURCE_SERVER || '|' || SOURCE_JOBID || '|' || TARGET_SERVER || '|' || TARGET_JOBID || '|' || to_char(JOB_INSERT_TIME,'yyyymmddHH24MISS'))";
var tab_key = Database.retrieve(sql);

if (tab_key != "")
{
sql = "select TARGET_JOBID from STG01.Z_DWH_SCHEDULER_STEUERUNG s where '" + tab_key + "' = (SOURCE_SERVER || '|' || SOURCE_JOBID || '|' || TARGET_SERVER || '|' || TARGET_JOBID || '|' || to_char(JOB_INSERT_TIME,'yyyymmddHH24MISS'))";
ret_val_str = Database.retrieve(sql);
job_id = ret_val_str;

sql = "update STG01.Z_DWH_SCHEDULER_STEUERUNG set JOB_STATUS=2, JOB_EXECUTE_TIME=sysdate where '" + tab_key + "' = (SOURCE_SERVER || '|' || SOURCE_JOBID || '|' || TARGET_SERVER || '|' || TARGET_JOBID || '|' || to_char(JOB_INSERT_TIME,'yyyymmddHH24MISS'))";
ret_val = Database.execute(sql);
sql = "commit";
ret_val = Database.execute(sql);

var internal_id = Scheduler.runJob(job_id);
}

Database.disconnect();


Thank you
Dominik
Wed Aug 06, 2014 4:27 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 6501

Post Reply with quote
How many records are returned by the database query?
Do you see jdbc.log in the 24x7 directory? If yes, is there any additional information in that related to that exception?
Wed Aug 06, 2014 8:07 am View user's profile Send private message
Dominik



Joined: 30 Jul 2007
Posts: 36

Post Reply with quote
Just one record with one value (jobid to start) or nothing.

in jdbc.log we can often find (before crash) something like that:



Wed Aug 06 14:51:00 CEST 2014
Job: 24x7 Scheduler
----------------------------------
registerDriver: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@324b85]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@fb0b0]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@a85268]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@83b18f]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@12b9f79]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@e7ccd7]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@161401f]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@137fea]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@113e371]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@1b6956f]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@ede19e]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@161b0bc]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@11641cf]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@1313e0e]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@16d079a]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@811e18]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@1346812]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@1a06ec2]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@111c3f0]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@e3f02a]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@1ac1adf]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@199049]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@90fd20]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@4ba66]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@d8355]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@4e4fc4]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@13a0067]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@6e056e]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@b1404f]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@148f7e0]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@8c1852]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@37086a]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@1a5ee7c]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@e51510]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@174e4b3]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@16d3d51]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@1afe17b]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@fef26a]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@6bc5aa]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@9b1ac]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@15660ef]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@23a559]
skipping: driver[className=oracle.jdbc.driver.OracleDriver,oracle.jdbc.driver.OracleDriver@6412da]
DriverManager.deregisterDriver: oracle.jdbc.OracleDriver@a998c1
couldn't find driver to unload




Scheduler hang up again at 14:57 (at this same time is no log in jdbc) , this time the last message in debug.log was:

2014-08-06 14:57:00,006 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): starting JavaScript
2014-08-06 14:57:00,006 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): job not using user authentication
2014-08-06 14:57:00,006 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - evaluateScript(): starting jsthread
2014-08-06 14:57:00,006 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - evaluateScript(): waiting
2014-08-06 14:57:00,006 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - jsthread started
2014-08-06 14:57:00,006 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptRunner - runScript: JavaScriptRunner
2014-08-06 14:57:00,006 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptRunner - Connecting to database...
2014-08-06 14:57:00,006 [JavaScript] DEBUG com.softtreetech.jscheduler.business.sql.SqlConnector - loadDriver() driverPath=./drivers/ora/ojdbc14_g.jar driverClass=oracle.jdbc.OracleDriver
2014-08-06 14:57:00,006 [JavaScript] DEBUG com.softtreetech.jscheduler.business.sql.SqlConnector - parsed urls: 1
2014-08-06 14:57:00,021 [JavaScript] DEBUG com.softtreetech.jscheduler.business.sql.SqlConnector - url #0 file:/C:/Program Files/24x7/drivers/ora/ojdbc14_g.jar
2014-08-06 14:57:00,021 [JavaScript] DEBUG com.softtreetech.jscheduler.business.sql.SqlConnector - java.net.URLClassLoader@19dd520
2014-08-06 14:57:00,583 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - java.lang.OutOfMemoryError: PermGen space
2014-08-06 14:57:00,583 [JavaScript] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - jsthread stopped
2014-08-06 14:57:00,583 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - evaluateScript(): script terminated with exception: java.lang.OutOfMemoryError: PermGen space
2014-08-06 14:57:00,583 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): script evaluated
2014-08-06 14:57:00,583 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): stopping local js thread
2014-08-06 14:57:00,583 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.javascript.JavaScriptThreadImpl - stopJScript()
2014-08-06 14:57:00,583 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.runner.JavaScriptJobRunner - runJob(): job finished
2014-08-06 14:57:00,599 [Job #45 - MSI Remote Control - einfach Pollen] DEBUG com.softtreetech.jscheduler.business.queue.JobQueue - QUEUE_UNLOCKED

Wed Aug 06, 2014 10:58 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 6501

Post Reply with quote
Please try modifying master.bat file or master.sh, whichever you use to start the scheduler and the line starting with Java, add at the end -XX:MaxPermSize=256m

This should increase the maximum allowed size of permanent heap memory.

if you run it as a service, the same would need to be added to the system registry as service parameters.
Wed Aug 06, 2014 1:19 pm View user's profile Send private message
Dominik



Joined: 30 Jul 2007
Posts: 36

Post Reply with quote
Hi,

i'll try it - thank you.

The job is just a Mini-(polling)Job whith no big data or other memory catching elements. So in my opinion it looks like a memory leak.
Is the parameter a solution or just a workaround, which extens the time until the problem will appear again?

Thank you
Dominik
Thu Aug 07, 2014 3:22 am View user's profile Send private message
Dominik



Joined: 30 Jul 2007
Posts: 36

Post Reply with quote
Sorry, but I still have the same Problem.
Fri Aug 08, 2014 9:21 am View user's profile Send private message
Dominik



Joined: 30 Jul 2007
Posts: 36

Post Reply with quote
I still have the same problem.
After 2-3h the 24x7 Scheduler hang up an has to be kill by taskmanager.

May it be the jdbc entry:
DriverManager.deregisterDriver: oracle.jdbc.OracleDriver@1a8e0bd
couldn't find driver to unload

??

I can't understand it. Driver is loading, SQL is executing. But the driver is not unloading.

Thank you
Dominik
Tue Aug 12, 2014 4:24 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 6501

Post Reply with quote
I suggest 2 step approach
1. Setting a timeout for that job, so that the scheduler can clear it from the queue and it won't block other jobs
2. Trying to figure out the route cause leading to the out of memory error. Can you reproduce the issue if you comment out var internal_id = Scheduler.runJob(job_id); line?
Tue Aug 12, 2014 2:53 pm View user's profile Send private message
Dominik



Joined: 30 Jul 2007
Posts: 36

Post Reply with quote
I've googled a little bit and at the moment i think it's a problem between your 24x7 scheduler as an 32bit application and our 64bit (oracle) odbc driver.
What do you think abaout?
Wed Aug 13, 2014 7:37 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 6501

Post Reply with quote
As far as I can see from your log, it doesn't use ODBC, it connects to your server directly using Oracle JDBC driver supplied with the scheduler.

One of the available options is to try upgrading to a more recent driver version, but it would make sense after we establish the issue is in the driver or between the driver and the scheduler. That's why I asked you to comment out "runJob" to exclude other factors. If after commenting "runJob" the issue is reproducible, then updating the driver would be next logical step, otherwise, it's something else.
Wed Aug 13, 2014 8:09 am View user's profile Send private message
Dominik



Joined: 30 Jul 2007
Posts: 36

Post Reply with quote
Quote:
That's why I asked you to comment out "runJob" to exclude other factors.


I did it, but still have the problem
Tue Aug 19, 2014 9:31 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 6501

Post Reply with quote
Thank you. This indicates that most likely the issue is caused by database operations. I suggest installing latest version of 24x7. latest versions are packaged with a different version of Oracle JDBC driver. Please install new version into a different folder, do not override your existing installation. Recreate DWH database connection settings. Recreate that particular job keeping runJob line commented out so as not to have the job to depend on any other job. Please try running it using the new version and see if that resolves the issue.
Tue Aug 19, 2014 9:33 pm View user's profile Send private message
Display posts from previous:    
Reply to topic    SoftTree Technologies Forum Index » 24x7 Scheduler, Event Server, Automation Suite All times are GMT - 4 Hours
Page 1 of 1

 
Jump to: 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


 

 

Powered by phpBB © 2001, 2005 phpBB Group
Design by Freestyle XL / Flowers Online.