ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)

Problem Statement:-

 I have just finished up with the SOA Suite 11.1.1.7 environment, and started the soa_server1 using ‘startManagedWebLogic.sh’ script and server bought up without any problem, but after shutting down when I tried starting through node managed it was keep failing and was giving the below error.

ERROR: transport error 202: bind failed: Address already in use

ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)

JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../../../src/share/back/debugInit.c:690]

FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197)

[ERROR] aborted

JRockit aborted: Unknown error (50)

<Jul 11, 2013 11:24:52 AM> <FINEST> <NodeManager> <Waiting for the process to die: 24746>

<Jul 11, 2013 11:24:52 AM> <INFO> <NodeManager> <Server failed during startup so will not be restarted>

<Jul 11, 2013 11:24:52 AM> <FINEST> <NodeManager> <runMonitor returned, setting finished=true and notifying waiters>

Solution:-

The error message was pretty clear that, while starting weblogic server is not able to bind with some port, now there were three challenges here-

1) What is the port which weblogic failed to bind?

2) Which PID is using that unidentified port?

3) Where to change that port value?

Port identification:-

Weblogic servers does not have much ports, as per my understanding as of now only few ports been used e.g. http listen port, ssl port, alsb debug port and java debug port. 

I was sure about http listen port and ssl configuration ports number for other servers on same machine, so there was no point to look into that area, but debug port is something I believe could cause the issue.

Looking at server start up parameter in .out file, I was able to figure out port 8453 was getting used, and I believe that was used for debugging purpose because error message says Java Debugging wire protocol (JDWP) failed to bind the port. 

So, port number been identified, next steps to check whether this port in is listening mode or not, its pretty straight forward to check the same, run the netstat command to check the same. Here is the command

“netstat –an | grep 8453’

You will see below kind of message-

tcp        0      0 0.0.0.0:8453                0.0.0.0:*                   LISTEN

So, its was confirmed this port was in use by some PID, next challenge was to findout the PID which was consuming this port. 

PID Identification:-

Goggling through internet I came across two commands which helps to check the PID based on port number in Linux environment, In my case I was using RED Hat Linux Enterprise (RHEL 5.8) version. 

Run the below command to find out used PID for these port.

Command 1: 

“lsof | grep 8453”

Below kind of message you will get in respons-

java      10387    fusion   10u     IPv4           14242572        0t0        TCP *:8453 (LISTEN)

In above message 10387 is the PID

Command2:

“fuser 8453/tcp” 

You will see below message-

8453/tcp:            10387

So 10387 is the port id.

Changing the port:-

So, now you know what is the port number, which PID is consuming this port, so next step is either kill that process which is consuming the port or change the port number to some other number for starting process/server. Its’ purely depend on your requirement and criticality of that PID which is already consuming the port 8453.

Run this command to know more about PID. 

“ps -ax | grep 10387”

You will see details about process id which will help you to decide either kill this process or change the port number to some other number for starting server which you are trying to start. 

In my case I have found that port was configured in setDomainEnv.sh file, since all the managed server in the domain which I have created were installed on same physical server, this conflict was coming. I have changed the setDomainEnv.sh file to consider different port number for all managed servers.