ARIS Community - We Love BPM

ARIS 10: Failed / Warning messages in runnables' logs after health check

Pedro Guerrero's picture
by Pedro Guerrero in Administration & User Management posted on 2018-01-31

Hello everyone.

First of all, I'm new to ARIS Administration so this question could be basic for many of you but at this point there is an issue that I need to fix so I'm here asking for any assistance that can help me address the issue properly. Thank you all in advance.

After performing a health check at our customer's ARIS Infrastructure I found one error and one warning as seen below:

 

  • For the failed runnable (ecp_l) I got this message: "FAILED Technical user not found for tenant id(s) : [default]"

 

From the error.log file I got this persistent message

2018-01-22 23:32:55,580|ERROR|ecp0000000000|default||0000000000|null-SimpleThreadPoolWorker-1|DataFeedGeneratorExecution - Document Storage user is not configured
2018-01-23 23:32:55,718|ERROR|ecp0000000000|default||0000000000|null-SimpleThreadPoolWorker-6|DataFeedGeneratorExecution - Document Storage user is not configured
2018-01-24 23:32:55,643|ERROR|ecp0000000000|default||0000000000|null-SimpleThreadPoolWorker-1|DataFeedGeneratorExecution - Document Storage user is not configured
2018-01-25 23:32:55,744|ERROR|ecp0000000000|default||0000000000|null-SimpleThreadPoolWorker-6|DataFeedGeneratorExecution - Document Storage user is not configured
2018-01-26 23:32:55,812|ERROR|ecp0000000000|default||0000000000|null-SimpleThreadPoolWorker-1|DataFeedGeneratorExecution - Document Storage user is not configured
2018-01-27 23:32:55,749|ERROR|ecp0000000000|default||0000000000|null-SimpleThreadPoolWorker-6|DataFeedGeneratorExecution - Document Storage user is not configured
2018-01-28 23:32:55,515|ERROR|ecp0000000000|default||0000000000|null-SimpleThreadPoolWorker-1|DataFeedGeneratorExecution - Document Storage user is not configured
2018-01-29 23:32:55,514|ERROR|ecp0000000000|default||0000000000|null-SimpleThreadPoolWorker-6|DataFeedGeneratorExecution - Document Storage user is not configured
2018-01-30 23:32:55,831|ERROR|ecp0000000000|default||0000000000|null-SimpleThreadPoolWorker-1|DataFeedGeneratorExecution - Document Storage user is not configured

As you can see, there would not be any relation between these two types of messages

  • For the runnable with the warning (apg_l) I got this one: "WARNING License is ARIS Cloud Advanced Server. Limited functionality only."

2018-01-25 09:14:48,616 Tomcat-http-13 ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-19.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-19.log: The process cannot access the file because it is being used by another process.

2018-01-25 09:14:48,616 Tomcat-http-13 ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

2018-01-27 01:29:04,623 AGE-Engine-ProcessScheduler_QuartzSchedulerThread ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-20.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-20.log: The process cannot access the file because it is being used by another process.

2018-01-27 01:29:04,623 AGE-Engine-ProcessScheduler_QuartzSchedulerThread ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

2018-01-28 00:04:31,658 AGE-TS-TaskNotificationsScheduler_QuartzSchedulerThread ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-21.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-21.log: The process cannot access the file because it is being used by another process.

2018-01-28 00:04:31,658 AGE-TS-TaskNotificationsScheduler_QuartzSchedulerThread ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

2018-01-29 03:34:48,889 AGE-TS-TaskNotificationsScheduler_QuartzSchedulerThread ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-22.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-22.log: The process cannot access the file because it is being used by another process.

2018-01-29 03:34:48,889 AGE-TS-TaskNotificationsScheduler_QuartzSchedulerThread ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

2018-01-30 05:59:51,643 Tomcat-http-25 ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-23.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-23.log: The process cannot access the file because it is being used by another process.

2018-01-30 05:59:51,643 Tomcat-http-25 ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

2018-01-31 08:02:37,575 AGE-Engine-ProcessScheduler_QuartzSchedulerThread ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-24.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-24.log: The process cannot access the file because it is being used by another process.

2018-01-31 08:02:37,575 AGE-Engine-ProcessScheduler_QuartzSchedulerThread ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

 

Again, I see no relationship between one message or another

 

 

Any idea on how to address this? Is there any knowledge base where I can match these error messages with possible root causes and solutions? (BTW, we're still managing to enable our Empower account as partners) I thank you in advance for the assistance since I need to have these runnables healthy for our client.

 

Best regards

Pedro.

644 Views
0 Likes
6 Comments
Sorry there are no tags
There are no attachments
Rune Becker posted on 2018-01-31

Dear Pedro,

You could open a support ticket in the name of your customer, collect the log files and provide it to Global Support ARIS via Empower.

For ARIS partners we offer a log file analysis tool. But since your Empower account is still in progress, give the support a call: https://empower.softwareag.com/public_directory.asp

Cheers
Rune

Pedro Guerrero posted on 2018-01-31

Hi Rune

I cannot give a call to Mexico at the customer's facility but I sent an e-mail to the empower mail address in the meantime.

Thank you

Pedro.

Pedro Guerrero posted on 2018-02-07

Here's what how I made it for the FAILED runnable:

 

Runnable with FAILED status was related to the following message:  Technical user not found for tenant ids : [default]

Logs excerpt containing the actual error message:

2018-02-07 12:17:16,392|INFO |ecp0000000000|default|UNWxhoTufQ|0000004318|Tomcat-ajp-7|ElasticSearchQueryBuilderForAuditlog - Search Query for tenant: default is "query":{"bool":{"must":[{"match":{"eventType":"TECHUSER_FOUND"}}]}},"sort":[{"createdTime":{"order":"desc","ignore_unmapped" : true}}],"size":1
2018-02-07 12:17:16,392|INFO |ecp0000000000|default|UNWxhoTufQ|0000004318|Tomcat-ajp-7|TechnicalUserCheck - Technical user not found for tenant ids : [default]
2018-02-07 12:17:34,365|INFO |ecp0000000000|default|UNWxhoTufQ|0000004319|Tomcat-ajp-8|ElasticSearchQueryBuilderForAuditlog - Elastic Search query formed for auditlog search is successful
2018-02-07 12:17:34,365|INFO |ecp0000000000|default|UNWxhoTufQ|0000004319|Tomcat-ajp-8|ElasticSearchQueryBuilderForAuditlog - Search Query for tenant: default is "query":{"bool":{"must":[{"match":{"eventType":"TECHUSER_FOUND"}}]}},"sort":[{"createdTime":{"order":"desc","ignore_unmapped" : true}}],"size":1
2018-02-07 12:17:34,365|INFO |ecp0000000000|default|UNWxhoTufQ|0000004319|Tomcat-ajp-8|TechnicalUserCheck - Technical user not found for tenant ids : [default]
2018-02-07 12:17:35,756|INFO |ecp0000000000|default|UNWxhoTufQ|0000004319|Tomcat-ajp-8|ElasticSearchQueryBuilderForAuditlog - Elastic Search query formed for auditlog search is successful
2018-02-07 12:17:35,756|INFO |ecp0000000000|default|UNWxhoTufQ|0000004319|Tomcat-ajp-8|ElasticSearchQueryBuilderForAuditlog - Search Query for tenant: default is "query":{"bool":{"must":[{"match":{"eventType":"TECHUSER_FOUND"}}]}},"sort":[{"createdTime":{"order":"desc","ignore_unmapped" : true}}],"size":1
2018-02-07 12:17:35,756|INFO |ecp0000000000|default|UNWxhoTufQ|0000004319|Tomcat-ajp-8|TechnicalUserCheck - Technical user not found for tenant ids : [default]
2018-02-07 12:33:16,314|INFO |ecp0000000000|default|5dB9_2GE89|0000004327|Tomcat-ajp-17|ElasticSearchQueryBuilderForAuditlog - Elastic Search query formed for auditlog search is successful
2018-02-07 12:33:16,314|INFO |ecp0000000000|default|5dB9_2GE89|0000004327|Tomcat-ajp-17|ElasticSearchQueryBuilderForAuditlog - Search Query for tenant: default is "query":{"bool":{"must":[{"match":{"eventType":"TECHUSER_FOUND"}}]}},"sort":[{"createdTime":{"order":"desc","ignore_unmapped" : true}}],"size":1
2018-02-07 12:33:16,314|INFO |ecp0000000000|default|5dB9_2GE89|0000004327|Tomcat-ajp-17|TechnicalUserCheck - Technical user not found for tenant ids : [default]
2018-02-07 12:33:17,705|INFO |ecp0000000000|default|5dB9_2GE89|0000004327|Tomcat-ajp-17|ElasticSearchQueryBuilderForAuditlog - Elastic Search query formed for auditlog search is successful
2018-02-07 12:33:17,705|INFO |ecp0000000000|default|5dB9_2GE89|0000004327|Tomcat-ajp-17|ElasticSearchQueryBuilderForAuditlog - Search Query for tenant: default is "query":{"bool":{"must":[{"match":{"eventType":"TECHUSER_FOUND"}}]}},"sort":[{"createdTime":{"order":"desc","ignore_unmapped" : true}}],"size":1
2018-02-07 12:33:17,705|INFO |ecp0000000000|default|5dB9_2GE89|0000004327|Tomcat-ajp-17|TechnicalUserCheck - Technical user not found for tenant ids : [default]
2018-02-07 12:39:35,563|INFO |ecp0000000000|default|Q-u2dT7qjw|0000004346|Tomcat-ajp-21|StreamDiscoverListsMapper - Finished generating Stream Discovery lists for all users.

Then I went to the Tenant Management web console and performed a default tenant backup

2018-02-07 12:42:40,952|INFO |ecp0000000000|default|CYcobyEe1Y|0000004398|Tomcat-http-19|TenantManagementResource - About to perform backup for default
2018-02-07 12:42:41,530|INFO |ecp0000000000|default||0000000000|main-EventThread|UmcNodeWatcher - tenant identifier default
2018-02-07 12:42:41,530|INFO |ecp0000000000|default||0000000000|main-EventThread|AddUpdatePeopleExecution - technical username _aris_tech_user_ecp_techuser
2018-02-07 12:42:41,530|INFO |ecp0000000000|default||0000000000|main-EventThread|TechnicalUserCheckExecution - Maintaining tech user state for tenant default - started
2018-02-07 12:42:41,530|INFO |ecp0000000000|default||0000000000|main-EventThread|ElasticSearchQueryBuilderForAuditlog - Elastic Search query formed for auditlog search is successful
2018-02-07 12:42:41,530|INFO |ecp0000000000|default||0000000000|main-EventThread|ElasticSearchQueryBuilderForAuditlog - Search Query for tenant: default is "query":{"bool":{"must":[{"match":{"eventType":"TECHUSER_FOUND"}}]}},"sort":[{"createdTime":{"order":"desc","ignore_unmapped" : true}}],"size":1

After backing up the tenant, then I got the following messages:  " Technical user found for tenant default"

2018-02-07 12:42:41,983|INFO |ecp0000000000|default||0000000000|main-EventThread|TechnicalUserCheckExecution - Technical user found for tenant default
2018-02-07 12:42:41,999|INFO |ecp0000000000|default||0000000000|main-EventThread|TechnicalUserCheckExecution - Maintaining tech user state for tenant default - completed 
2018-02-07 12:42:41,999|INFO |ecp0000000000|default||0000000000|main-EventThread|AddUpdatePeopleExecution - technical user node data changed
2018-02-07 12:42:42,547|INFO |ecp0000000000|default|CYcobyEe1Y|0000004398|Tomcat-http-19|RoundRobinServiceInstance - isDBUnifiedSchema : true
2018-02-07 12:42:42,907|INFO |ecp0000000000|default|CYcobyEe1Y|0000004398|Tomcat-http-19|Schema - Total number of table collected is 84
2018-02-07 12:42:42,907|INFO |ecp0000000000|default|CYcobyEe1Y|0000004398|BackupOperation-default-58919dbd-06e7-4163-803d-964e916d8c5e|ProgressManager - The current progress is preBackupOperationPGR

Then I ran another health check on the runnable and passed OK. 

I'm not sure how can I relate the tenant backup with the lack of a technical user for the tenant but the runnable is no longer failed.

Hope this helps

Regards

Pedro.

Rune Becker posted on 2018-02-07

Dear Pedro,

The missing technical user isn't an error but a warning.

But as you have installed "l" size of ARIS, is there sufficient amount of RAM on the computer/VM?

Cheers
Rune

Pedro Guerrero posted on 2018-02-07

As per the ARIS System Requirements we have the recommended hardware requirements for this server

36 GB (free RAM)

>= 16 CPU Cores

>= 100GB free HD Space

 

So far there are few active users so there would be enough RAM at this point.

Regards

Pedro.

 

 

Pedro Guerrero posted on 2018-02-14

As for the apg_l runnable, we still have the WARNING state when performing a health check over it. I have noticed that there is a persistent log message referring to this path: E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log

The thing is that it looks like it's a scheduled task trying to move a log file from one place to another. It appears once a day according to the following log trace:

 

2018-02-10 09:30:38,442 AGE-TS-TaskNotificationsScheduler_QuartzSchedulerThread ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: The process cannot access the file because it is being used by another process.

2018-02-10 09:30:38,442 AGE-TS-TaskNotificationsScheduler_QuartzSchedulerThread ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

2018-02-11 10:25:03,523 Tomcat-http-22 ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: The process cannot access the file because it is being used by another process.

2018-02-11 10:25:03,523 Tomcat-http-22 ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

2018-02-12 02:40:05,410 Tomcat-http-18 ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: The process cannot access the file because it is being used by another process.

2018-02-12 02:40:05,426 Tomcat-http-18 ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

2018-02-13 04:27:32,721 AGE-TS-TaskNotificationsScheduler_QuartzSchedulerThread ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: The process cannot access the file because it is being used by another process.

2018-02-13 04:27:32,721 AGE-TS-TaskNotificationsScheduler_QuartzSchedulerThread ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.

2018-02-14 01:36:06,024 Tomcat-http-17 ERROR Unable to move file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log to E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log -> E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age-2017-11-10-25.log: The process cannot access the file because it is being used by another process.

2018-02-14 01:36:06,024 Tomcat-http-17 ERROR Unable to delete file E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: java.nio.file.FileSystemException E:\SoftwareAG\ARIS10.0\server\bin\work\work_apg_l\base\logs\age.log: The process cannot access the file because it is being used by another process.
 

i've been reviewing if there are any pending governance process but I can't find one that leads me to the answer.

Is anybody familiar with these type of messages? please let me know

Regards

Pedro.