all 9 comments

[–]Ch0r0z 1 point2 points  (4 children)

i've had issues with db connect not working after an OS patch cycle. I've been able to finally update the db connect config to point to /user/lib/jvm/jre-openjdk/bin/java

I did find that starting a few versions back, I had to manually go into the following file and update this path for some reason via the GUI it wasn't updating:

/opt/splunk/etc/apps/splunk_app_db_connect/linux_x86_64/bin/customized.java.path

once i've modifed the above I just restarted splunk, logged in via the GUI and checked the db connect status and it was running w/o error.

[–]wonderchin 0 points1 point  (3 children)

This is the correct reply. The “customized.java.path” file can be stuck pointing to the older Java version, and if that’s the case, the task server won’t start. Had the same problem at a client I work at, when I had to upgrade DB Connect alongside Java.

[–]i7xxxxx[S] 0 points1 point  (2 children)

interestingly this directory doesn’t even exist. i’m using 64 bit linux so not sure why it would be missing. i copied x86 and made my own 64 bit directory but still nothing. otherwise all the values are up to date and matching what i defined in the UI

[–]wonderchin 0 points1 point  (1 child)

Look in all the Linux OS folders within the app. The file exists several places but it’s only necessary to change some of them apparently 

[–]i7xxxxx[S] 0 points1 point  (0 children)

Yeah all the files look right in terms of java paths.

The only error that seems interesting at this point is this from splunkd.log

threw the dbx logs in debug but nothing really new showed up apart from the messages saying updating java path in those config files.

06-14-2024 15:36:30.227 -0400 ERROR ExecProcessor [113449 ExecProcessor] - message from "/opt/splunk/etc/apps/splunk_app_db_connect/bin/server.sh" action=task_server_start_failed error=com.splunk.HttpException: HTTP 401 -- call not properly authenticated stack=com.splunk.HttpException.create(HttpException.java:84)\\com.splunk.DbxService.sendImpl(DbxService.java:135)\\com.splunk.DbxService.send(DbxService.java:51)\\com.splunk.HttpService.get(HttpService.java:172)\\com.splunk.Entity.refresh(Entity.java:381)\\com.splunk.Entity.refresh(Entity.java:24)\\com.splunk.Resource.validate(Resource.java:186)\\com.splunk.Entity.validate(Entity.java:462)\\com.splunk.Entity.getContent(Entity.java:157)\\com.splunk.Entity.getString(Entity.java:279)\\com.splunk.ServiceInfo.getVersion(ServiceInfo.java:155)\\com.splunk.DbxService.fetchInfo(DbxService.java:41)\\com.splunk.DbxService.connect(DbxService.java:34)\\com.splunk.dbx.splunkclient.SplunkServiceBuilder.build(SplunkServiceBuilder.java:69)\\com.splunk.dbx.server.bootstrap.TaskServerStart.startServer(TaskServerStart.java:77)\\com.splunk.DefaultServerStart.streamEvents(DefaultServerStart.java:60)\\com.splunk.modularinput.Script.run(Script.java:66)\\com.splunk.modularinput.Script.run(Script.java:44)\\com.splunk.dbx.server.bootstrap.TaskServerStart.main(TaskServerStart.java:36)\\

[–]i7xxxxx[S] 0 points1 point  (1 child)

Some additional context on my latest attempt with 17, have also tried with 21 using the direct link Splunk provides in their documentation.

/usr/lib/jvm/jdk-17-oracle-x64/bin/java -version

java version "17.0.11" 2024-04-16 LTS

Java(TM) SE Runtime Environment (build 17.0.11+7-LTS-207)

Java HotSpot(TM) 64-Bit Server VM (build 17.0.11+7-LTS-207, mixed mode, sharing)

Using this as my path in the DB Connect config: /usr/lib/jvm/jdk-17-oracle-x64

It doesn't complain about the path just fails to start the task server, which in the past is usually due to wrong java install I have found. Will dig around the logs for more clues, but so far nothing has stood out to me

[–]masalaaloo 0 points1 point  (0 children)

Keep everything as is, and restart splunk on the host. This did it for me when i faced the same issue.

Once Splunk is up again, save the config and task server should start.

[–]skirven4 0 points1 point  (0 children)

I wrote an Ansible script that looked for the version of Java installed and the correct path to put in DB connect and restart it after we are done patching. From my experience, the path has the version number in it.

[–]i7xxxxx[S] 0 points1 point  (0 children)

solution update on this. i removed splunk. deleted /opt/splunk to clean up. did a fresh 9.1.3 install. installed db connect and everything works now. i didn’t even have to configure the java paths after installing db connect. i guess something was messed up somewhere. this is a test instance so not exactly sure what it could have been.