Sunday, August 2, 2015

SOA_INFRA - Cluster communication initialization failed

Cluster communication initialization failed : 


More details From the Log File:

<Jul 28, 2015 11:38:18 PM PDT> <Error> <oracle.integration.platform.blocks.deploy.coordinator> <BEA-000000> <Cluster communication initialization failed. If you are using multicast, please make sure multicast is enabled on your network and that there is no interference on the address in use. Please see the documentation for more details.

(Wrapped) java.net.BindException: Address already in use

<Jul 28, 2015 11:38:18 PM PDT> <Error> <Coherence> <BEA-000000> <2015-07-28 23:38:18.531/921.992 Oracle Coherence GE 3.7.1.1 <Error> (thread=[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)', member=n/a): Error while starting cluster: (Wrapped) java.net.BindException: Address already in use
at com.tangosol.util.Base.ensureRuntimeException(Base.java:288)
at com.tangosol.util.Base.ensureRuntimeException(Base.java:269)
at com.tangosol.coherence.component.util.SafeCluster.startCluster(SafeCluster.CDB:3)
at com.tangosol.coherence.component.util.SafeCluster.restartCluster(SafeCluster.CDB:10)
at com.tangosol.coherence.component.util.SafeCluster.ensureRunningCluster(SafeCluster.CDB:26)
at com.tangosol.coherence.component.util.SafeCluster.start(SafeCluster.CDB:2)

Impact: SOA_INFRA is running only on one MS server when we have 4 Managed servers(Cluster) and also we are not able to see from wwwin.test.world.com/em


Under SOA_INFRA - only 1 Managed server is visible.

Description

Coherence is an in-memory data grid which helps eliminate single points of failure and single points of bottlenecks in an application by distributing application objects and logic across multiple physical servers.


Oracle Coherence is the industry leading in-memory data grid solution that enables organizations to predictably scale mission-critical applications by providing fast access to frequently used data.


– Coherence is bundled with WLS 10.3.3 and above.
.
– Make sure you download the Package Installer for WLS which includes Coherence.

Example: The installation of WebLogic Server 11gR1 (10.3.3) also installs Coherence 3.5 in the coherence_3.5 directory



Important terms :

Coherence Node – a Java process – it is an instance of coherence server – has well known ports and address to communicate with other nodes.
Coherence Cluster – Group of storage enabled cluster nodes – specified by a common cluster name
Coherence Client – Storage disabled nodes.do not hold data but perform data operations
Coherence Server – Storage enabled nodes. Hold Cache Data and performs server processing
Cache – Local memory space that holds frequently requested data


Fix : wka cluster settings were missing after adding issue fixed.

Parameter added under setDomainenv.sh of mserver : 

-Dsoa.instance.home=${DOMAIN_HOME} -XX:+HeapDumpOnOutOfMemoryError -Dtangosol.coherence.wka1=a1-test-119 -Dtangosol.coherence.wka2=a2-test-120 -Dtangosol.coherence.wka3=a3-test-13 -Dtangosol.coherence.wka4=a4-test-14 -Dtangosol.coherence.localhost=$HOST_NAME -Dtangosol.coherence.localport=9091 -Dtangosol.coherence.wka1.port=9091 -Dtangosol.coherence.wka2.port=9091 -Dtangosol.coherence.wka3.port=9091 -Dtangosol.coherence.wka4.port=9091