[Freeipa-users] Could not connect to LDAP server host - IO Error creating JSS SSL Socket:

[Michael Plemmons]

I have a three node IPA cluster.

ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt

ipa13 has agreements with ipa11 and ipa12. ipa11 and ipa12 do not have
agreements between each other.

It appears that either ipa12.mgmt lost some level of its replication
agreement with ipa13. I saw some level because users / hosts were
replicated between all systems but we started seeing DNS was not resolving
properly from ipa12. I do not know when this started.

When looking at replication agreements on ipa12 I did not see any agreement
with ipa13.

When I run ipa-replica-manage list all three hosts show has master.

When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt is a replica.

When I run ipa-replica-manage ipa12.mgmt nothing returned.

I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com ipa13.mgmt.crosschx.com on ipa12.mgmt

I then ran the following

ipa-replica-manage force-sync --from ipa13.mgmt.crosschx.com

ipa-replica-manage re-initialize --from ipa13.mgmt.crosschx.com

I was still seeing bad DNS returns when dig'ing against ipa12.mgmt. I was
able to create user and DNS records and see the information replicated
properly across all three nodes.

I then ran ipactl stop on ipa12.mgmt and then ipactl start on ipa12.mgmt
because I wanted to make sure everything was running fresh after the
changes above. While IPA was staring up (DNS started) we were able to see
valid DNS queries returned but pki-tomcat would not start.

I am not sure what I need to do in order to get this working. I have
included the output of certutil and getcert below from all three servers as
well as the debug output for pki.


While the IPA system is coming up I am able to successfully run ldapsearch
-x as the root user and see results. I am also able to login with the
"cn=Directory Manager" account and see results.


The debug log shows the following error.


[03/May/2017:21:22:01][localhost-startStop-1]:
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: ===== DEBUG SUBSYSTEM
INITIALIZED =======
[03/May/2017:21:22:01][localhost-startStop-1]:
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look for
cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=debug
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized debug
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look for
cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look for
cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: DBSubsystem: init()
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
[03/May/2017:21:22:01][localhost-startStop-1]: LdapBoundConnFactory: init
[03/May/2017:21:22:01][localhost-startStop-1]:
LdapBoundConnFactory:doCloning true
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init()
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init begins
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init ends
[03/May/2017:21:22:01][localhost-startStop-1]: init: before makeConnection
errorIfDown is true
[03/May/2017:21:22:01][localhost-startStop-1]: makeConnection: errorIfDown
true
[03/May/2017:21:22:02][localhost-startStop-1]:
SSLClientCertificateSelectionCB: Setting desired cert nickname to:
subsystemCert cert-pki-ca
[03/May/2017:21:22:02][localhost-startStop-1]: LdapJssSSLSocket: set client
auth cert nickname subsystemCert cert-pki-ca
[03/May/2017:21:22:02][localhost-startStop-1]:
SSLClientCertificatSelectionCB: Entering!
[03/May/2017:21:22:02][localhost-startStop-1]:
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host ipa12.mgmt.crosschx.com port 636
Error netscape.ldap.LDAPException: Authentication failed (48)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:166)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:130)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1169)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at
com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)
at
org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)
at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
at
org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at
org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)
at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect to LDAP server host
ipa12.mgmt.crosschx.com port 636 Error netscape.ldap.LDAPException:
Authentication failed (48)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1169)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at
com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)
at
org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)
at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
at
org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at
org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)
at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[03/May/2017:21:22:02][localhost-startStop-1]: CMSEngine.shutdown()


=============================


IPA11.MGMT


(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/

Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI

Server-Cert
u,u,uMGMT.CROSSCHX.COM IPA CA
CT,C,C

(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/

Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI

caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u





IPA13.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/

Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI

Server-Cert
u,u,uMGMT.CROSSCHX.COM IPA CA
CT,C,C

(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/

Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI

caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u




IPA12.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/

Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI

Server-Cert
u,u,uMGMT.CROSSCHX.COM IPA CA C,,

(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/

Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI

caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u

=================================================

IPA11.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
Request ID '20161229155314':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:52:43 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command:
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
Request ID '20161229155652':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229155654':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229155655':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229155657':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229155659':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 15:56:20 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229155921':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:52:46 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command:
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
Request ID '20161229160009':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes




==================================

IPA13.MGMT

(root)>getcert list
Number of certificates and requests being tracked: 8.
Request ID '20161229143449':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 14:34:20 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command:
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
Request ID '20161229143826':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229143828':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229143831':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229143833':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229143835':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 14:37:54 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229144057':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 14:34:23 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command:
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
Request ID '20161229144146':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes



===========================

IPA12.MGMT

(root)>getcert list
Number of certificates and requests being tracked: 8.
Request ID '20161229151518':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:14:51 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command:
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
Request ID '20161229151850':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229151852':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229151854':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229151856':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229151858':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 15:18:16 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20161229152115':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:14:54 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command:
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
Request ID '20161229152204':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes




*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

[Michael Plemmons]

I realized that I was not very clear in my statement about testing with
ldapsearch. I had initially run it without logging in with a DN. I was
just running the local ldapsearch -x command. I then tested on ipa12.mgmt
and ipa11.mgmt logging in with a full DN for the admin and "cn=Directory
Manager" from ipa12.mgmt (broken server) and ipa11.mgmt and both ldapsearch
command succeeded.

I ran the following from ipa12.mgmt and ipa11.mgmt as a non root user. I
also ran the command showing a line count for the output and the line
counts for each were the same when run from ipa12.mgmt and ipa11.mgmt.

ldapsearch -LLL -h ipa12.mgmt.crosschx.com -D "DN" -w PASSWORD -b
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn

ldapsearch -LLL -h ipa12.mgmt.crosschx.com -D "cn=directory manager" -w
PASSWORD dn






*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons <

Post by Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and ipa12 do not have
agreements between each other.
It appears that either ipa12.mgmt lost some level of its replication
agreement with ipa13. I saw some level because users / hosts were
replicated between all systems but we started seeing DNS was not resolving
properly from ipa12. I do not know when this started.
When looking at replication agreements on ipa12 I did not see any
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show has master.
When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt is a replica.
When I run ipa-replica-manage ipa12.mgmt nothing returned.
I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com ipa13.mgmt.crosschx.com on ipa12.mgmt
I then ran the following
ipa-replica-manage force-sync --from ipa13.mgmt.crosschx.com
ipa-replica-manage re-initialize --from ipa13.mgmt.crosschx.com
I was still seeing bad DNS returns when dig'ing against ipa12.mgmt. I was
able to create user and DNS records and see the information replicated
properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl start on ipa12.mgmt
because I wanted to make sure everything was running fresh after the
changes above. While IPA was staring up (DNS started) we were able to see
valid DNS queries returned but pki-tomcat would not start.
I am not sure what I need to do in order to get this working. I have
included the output of certutil and getcert below from all three servers as
well as the debug output for pki.
While the IPA system is coming up I am able to successfully run ldapsearch
-x as the root user and see results. I am also able to login with the
"cn=Directory Manager" account and see results.
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: ===== DEBUG SUBSYSTEM
INITIALIZED =======
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look
for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=debug
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized debug
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look
for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look
for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: DBSubsystem: init()
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
[03/May/2017:21:22:01][localhost-startStop-1]: LdapBoundConnFactory: init
LdapBoundConnFactory:doCloning true
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init()
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init begins
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init ends
[03/May/2017:21:22:01][localhost-startStop-1]: init: before
makeConnection errorIfDown is true
errorIfDown true
subsystemCert cert-pki-ca
[03/May/2017:21:22:02][localhost-startStop-1]: LdapJssSSLSocket: set
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host ipa12.mgmt.crosschx.com port 636
Error netscape.ldap.LDAPException: Authentication failed (48)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(
LdapBoundConnFactory.java:205)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(
LdapBoundConnFactory.java:166)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(
LdapBoundConnFactory.java:130)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(
CMSEngine.java:1169)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(
CMSEngine.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(
SecurityUtil.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(
SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at org.apache.catalina.security.SecurityUtil.execute(
SecurityUtil.java:320)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(
SecurityUtil.java:175)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(
SecurityUtil.java:124)
at org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1270)
at org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1195)
at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1085)
at org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5318)
at org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5610)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:899)
at org.apache.catalina.core.ContainerBase.access$000(
ContainerBase.java:133)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(
ContainerBase.java:156)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(
ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(
ContainerBase.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.
call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect to LDAP server host
Authentication failed (48)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(
CMSEngine.java:1169)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(
CMSEngine.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at com.netscape.cms.servlet.base.CMSStartServlet.init(
CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(
SecurityUtil.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(
SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at org.apache.catalina.security.SecurityUtil.execute(
SecurityUtil.java:320)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(
SecurityUtil.java:175)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(
SecurityUtil.java:124)
at org.apache.catalina.core.StandardWrapper.initServlet(
StandardWrapper.java:1270)
at org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1195)
at org.apache.catalina.core.StandardWrapper.load(
StandardWrapper.java:1085)
at org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:5318)
at org.apache.catalina.core.StandardContext.startInternal(
StandardContext.java:5610)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:899)
at org.apache.catalina.core.ContainerBase.access$000(
ContainerBase.java:133)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(
ContainerBase.java:156)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(
ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(
ContainerBase.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(
HostConfig.java:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.
run(HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.
call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[03/May/2017:21:22:02][localhost-startStop-1]: CMSEngine.shutdown()
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,uMGMT.CROSSCHX.COM IPA CA CT,C,C
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u
IPA13.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,uMGMT.CROSSCHX.COM IPA CA CT,C,C
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u
IPA12.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,uMGMT.CROSSCHX.COM IPA CA C,,
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u
=================================================
IPA11.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:52:43 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 15:56:20 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:52:46 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
==================================
IPA13.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 14:34:20 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 14:37:54 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 14:34:23 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
===========================
IPA12.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:14:51 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 15:18:16 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:14:54 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com

[Michael Plemmons]

I ran another test. I started IPA with the ignore service failure option
and I tired doing ldap searches like this.

ldapsearch -H ldaps://ipa12.mgmt.crosschx.com

from both my laptop and from ipa11.mgmt and I get successful returns when
logging in as the admin user and as the directory manager.

I then looked closer at the LDAP access logs for the last time I tried to
start up PKI and got the auth failure and i see this.


[04/May/2017:02:22:45.859021005 +0000] conn=12 fd=101 slot=101 SSL
connection from 10.71.100.92 to 10.71.100.92
[04/May/2017:02:22:45.875672450 +0000] conn=12 TLS1.2 256-bit AES
[04/May/2017:02:22:45.940908536 +0000] conn=12 op=0 BIND dn="" method=sasl
version=3 mech=EXTERNAL
[04/May/2017:02:22:45.942441120 +0000] conn=12 op=0 RESULT err=48 tag=97
nentries=0 etime=0

Is dn="" supposed to be empty?






*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

On Wed, May 3, 2017 at 10:16 PM, Michael Plemmons <

Post by Michael Plemmons
I realized that I was not very clear in my statement about testing with
ldapsearch. I had initially run it without logging in with a DN. I was
just running the local ldapsearch -x command. I then tested on ipa12.mgmt
and ipa11.mgmt logging in with a full DN for the admin and "cn=Directory
Manager" from ipa12.mgmt (broken server) and ipa11.mgmt and both ldapsearch
command succeeded.
I ran the following from ipa12.mgmt and ipa11.mgmt as a non root user. I
also ran the command showing a line count for the output and the line
counts for each were the same when run from ipa12.mgmt and ipa11.mgmt.
ldapsearch -LLL -h ipa12.mgmt.crosschx.com -D "DN" -w PASSWORD -b
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
ldapsearch -LLL -h ipa12.mgmt.crosschx.com -D "cn=directory manager" -w
PASSWORD dn
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com
On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons <

Post by Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and ipa12 do not have
agreements between each other.
It appears that either ipa12.mgmt lost some level of its replication
agreement with ipa13. I saw some level because users / hosts were
replicated between all systems but we started seeing DNS was not resolving
properly from ipa12. I do not know when this started.
When looking at replication agreements on ipa12 I did not see any
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show has master.
When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt is a replica.
When I run ipa-replica-manage ipa12.mgmt nothing returned.
I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com ipa13.mgmt.crosschx.com on ipa12.mgmt
I then ran the following
ipa-replica-manage force-sync --from ipa13.mgmt.crosschx.com
ipa-replica-manage re-initialize --from ipa13.mgmt.crosschx.com
I was still seeing bad DNS returns when dig'ing against ipa12.mgmt. I
was able to create user and DNS records and see the information replicated
properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl start on ipa12.mgmt
because I wanted to make sure everything was running fresh after the
changes above. While IPA was staring up (DNS started) we were able to see
valid DNS queries returned but pki-tomcat would not start.
I am not sure what I need to do in order to get this working. I have
included the output of certutil and getcert below from all three servers as
well as the debug output for pki.
While the IPA system is coming up I am able to successfully run
ldapsearch -x as the root user and see results. I am also able to login
with the "cn=Directory Manager" account and see results.
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: ===== DEBUG SUBSYSTEM
INITIALIZED =======
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look
for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=debug
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized debug
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look
for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look
for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: DBSubsystem: init()
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
[03/May/2017:21:22:01][localhost-startStop-1]: LdapBoundConnFactory: init
LdapBoundConnFactory:doCloning true
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init()
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init begins
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init ends
[03/May/2017:21:22:01][localhost-startStop-1]: init: before
makeConnection errorIfDown is true
errorIfDown true
subsystemCert cert-pki-ca
[03/May/2017:21:22:02][localhost-startStop-1]: LdapJssSSLSocket: set
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host ipa12.mgmt.crosschx.com port 636
Error netscape.ldap.LDAPException: Authentication failed (48)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConne
ction(LdapBoundConnFactory.java:205)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap
BoundConnFactory.java:166)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap
BoundConnFactory.java:130)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.
java:1169)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine
.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil
.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil
.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at org.apache.catalina.security.SecurityUtil.execute(SecurityUt
il.java:320)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu
rityUtil.java:175)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu
rityUtil.java:124)
at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1270)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1195)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1085)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5318)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5610)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:899)
at org.apache.catalina.core.ContainerBase.access$000(ContainerB
ase.java:133)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.
run(ContainerBase.java:156)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.
run(ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.
java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(
Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect to LDAP server
Authentication failed (48)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.
java:1169)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine
.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil
.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil
.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at org.apache.catalina.security.SecurityUtil.execute(SecurityUt
il.java:320)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu
rityUtil.java:175)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu
rityUtil.java:124)
at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1270)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1195)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1085)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5318)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5610)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:899)
at org.apache.catalina.core.ContainerBase.access$000(ContainerB
ase.java:133)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.
run(ContainerBase.java:156)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.
run(ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.
java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(
Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[03/May/2017:21:22:02][localhost-startStop-1]: CMSEngine.shutdown()
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,uMGMT.CROSSCHX.COM IPA CA CT,C,C
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u
IPA13.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,uMGMT.CROSSCHX.COM IPA CA CT,C,C
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u
IPA12.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,uMGMT.CROSSCHX.COM IPA CA C,,
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u
=================================================
IPA11.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:52:43 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 15:56:20 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:52:46 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
==================================
IPA13.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 14:34:20 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 14:37:54 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 14:34:23 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
===========================
IPA12.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:14:51 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 15:18:16 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:14:54 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com

[Michael Plemmons]

I also looked at RUVs and here is what I found. I do not know if anything
here is helpful.

ldapsearch -ZZ -h ipa11.mgmt.crosschx.com -D "cn=Directory Manager" -W -b
"o=ipaca"
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
| grep "nsds50ruv\|nsDS5ReplicaId"
nsDS5ReplicaId: 1095
nsds50ruv: {replicageneration} 58344598000000600000
nsds50ruv: {replica 1095 ldap://ipa11.mgmt.crosschx.com:389}
5865323f000004470
nsds50ruv: {replica 86 ldap://ipa13.mgmt.crosschx.com:389}
58651fdb00000056000
nsds50ruv: {replica 96 ldap://ipa11.mgmt.crosschx.com:389}
5834459c00000060000
nsds50ruv: {replica 91 ldap://ipa13.mgmt.crosschx.com:389}
583449970000005b000
nsds50ruv: {replica 97 ldap://ipa12.mgmt.crosschx.com:389}
583445c300000061000
nsds50ruv: {replica 81 ldap://ipa12.mgmt.crosschx.com:389}
5865295600000051000

IPA12 - this is the problem node.
ldapsearch -ZZ -h ipa12.mgmt.crosschx.com -D "cn=Directory Manager" -W -b
"o=ipaca"
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
| grep "nsds50ruv\|nsDS5ReplicaId"
nsDS5ReplicaId: 81
nsds50ruv: {replicageneration} 58344598000000600000
nsds50ruv: {replica 81 ldap://ipa12.mgmt.crosschx.com:389}
5865295600000051000
nsds50ruv: {replica 96 ldap://ipa11.mgmt.crosschx.com:389}
5834459c00000060000
nsds50ruv: {replica 86 ldap://ipa13.mgmt.crosschx.com:389}
58651fdb00000056000
nsds50ruv: {replica 91 ldap://ipa13.mgmt.crosschx.com:389}
583449970000005b000
nsds50ruv: {replica 97 ldap://ipa12.mgmt.crosschx.com:389}
583445c300000061000

ldapsearch -ZZ -h ipa13.mgmt.crosschx.com -D "cn=Directory Manager" -W -b
"o=ipaca"
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
| grep "nsds50ruv\|nsDS5ReplicaId"
nsDS5ReplicaId: 86
nsds50ruv: {replicageneration} 58344598000000600000
nsds50ruv: {replica 86 ldap://ipa13.mgmt.crosschx.com:389}
58651fdb00000056000
nsds50ruv: {replica 1095 ldap://ipa11.mgmt.crosschx.com:389}
5865323f000004470
nsds50ruv: {replica 96 ldap://ipa11.mgmt.crosschx.com:389}
5834459c00000060000
nsds50ruv: {replica 91 ldap://ipa13.mgmt.crosschx.com:389}
583449970000005b000
nsds50ruv: {replica 97 ldap://ipa12.mgmt.crosschx.com:389}
583445c300000061000
nsds50ruv: {replica 81 ldap://ipa12.mgmt.crosschx.com:389}
5865295600000051000





*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

On Wed, May 3, 2017 at 10:52 PM, Michael Plemmons <

Post by Michael Plemmons
I ran another test. I started IPA with the ignore service failure option
and I tired doing ldap searches like this.
ldapsearch -H ldaps://ipa12.mgmt.crosschx.com
from both my laptop and from ipa11.mgmt and I get successful returns when
logging in as the admin user and as the directory manager.
I then looked closer at the LDAP access logs for the last time I tried to
start up PKI and got the auth failure and i see this.
[04/May/2017:02:22:45.859021005 +0000] conn=12 fd=101 slot=101 SSL
connection from 10.71.100.92 to 10.71.100.92
[04/May/2017:02:22:45.875672450 +0000] conn=12 TLS1.2 256-bit AES
[04/May/2017:02:22:45.940908536 +0000] conn=12 op=0 BIND dn=""
method=sasl version=3 mech=EXTERNAL
[04/May/2017:02:22:45.942441120 +0000] conn=12 op=0 RESULT err=48 tag=97
nentries=0 etime=0
Is dn="" supposed to be empty?
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com
On Wed, May 3, 2017 at 10:16 PM, Michael Plemmons <

Post by Michael Plemmons
I realized that I was not very clear in my statement about testing with
ldapsearch. I had initially run it without logging in with a DN. I was
just running the local ldapsearch -x command. I then tested on ipa12.mgmt
and ipa11.mgmt logging in with a full DN for the admin and "cn=Directory
Manager" from ipa12.mgmt (broken server) and ipa11.mgmt and both ldapsearch
command succeeded.
I ran the following from ipa12.mgmt and ipa11.mgmt as a non root user. I
also ran the command showing a line count for the output and the line
counts for each were the same when run from ipa12.mgmt and ipa11.mgmt.
ldapsearch -LLL -h ipa12.mgmt.crosschx.com -D "DN" -w PASSWORD -b
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
ldapsearch -LLL -h ipa12.mgmt.crosschx.com -D "cn=directory manager" -w
PASSWORD dn
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com
On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons <

Post by Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and ipa12 do not have
agreements between each other.
It appears that either ipa12.mgmt lost some level of its replication
agreement with ipa13. I saw some level because users / hosts were
replicated between all systems but we started seeing DNS was not resolving
properly from ipa12. I do not know when this started.
When looking at replication agreements on ipa12 I did not see any
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show has master.
When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt is a replica.
When I run ipa-replica-manage ipa12.mgmt nothing returned.
I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com ipa13.mgmt.crosschx.com on ipa12.mgmt
I then ran the following
ipa-replica-manage force-sync --from ipa13.mgmt.crosschx.com
ipa-replica-manage re-initialize --from ipa13.mgmt.crosschx.com
I was still seeing bad DNS returns when dig'ing against ipa12.mgmt. I
was able to create user and DNS records and see the information replicated
properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl start on ipa12.mgmt
because I wanted to make sure everything was running fresh after the
changes above. While IPA was staring up (DNS started) we were able to see
valid DNS queries returned but pki-tomcat would not start.
I am not sure what I need to do in order to get this working. I have
included the output of certutil and getcert below from all three servers as
well as the debug output for pki.
While the IPA system is coming up I am able to successfully run
ldapsearch -x as the root user and see results. I am also able to login
with the "cn=Directory Manager" account and see results.
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: ===== DEBUG SUBSYSTEM
INITIALIZED =======
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look
for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=debug
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized debug
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look
for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: autoShutdown
crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to look
for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initSubsystem
id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to init
id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: DBSubsystem: init()
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
[03/May/2017:21:22:01][localhost-startStop-1]: LdapBoundConnFactory: init
LdapBoundConnFactory:doCloning true
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init()
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init begins
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init ends
[03/May/2017:21:22:01][localhost-startStop-1]: init: before
makeConnection errorIfDown is true
errorIfDown true
subsystemCert cert-pki-ca
[03/May/2017:21:22:02][localhost-startStop-1]: LdapJssSSLSocket: set
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host ipa12.mgmt.crosschx.com port 636
Error netscape.ldap.LDAPException: Authentication failed (48)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConne
ction(LdapBoundConnFactory.java:205)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap
BoundConnFactory.java:166)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap
BoundConnFactory.java:130)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.
java:1169)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine
.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil
.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil
.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at org.apache.catalina.security.SecurityUtil.execute(SecurityUt
il.java:320)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu
rityUtil.java:175)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu
rityUtil.java:124)
at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1270)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1195)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1085)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5318)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5610)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.j
ava:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:899)
at org.apache.catalina.core.ContainerBase.access$000(ContainerB
ase.java:133)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru
n(ContainerBase.java:156)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru
n(ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.
java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(Executor
s.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect to LDAP server
host ipa12.mgmt.crosschx.com port 636 Error
netscape.ldap.LDAPException: Authentication failed (48)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.
java:1169)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine
.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS
ervlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil
.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil
.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at org.apache.catalina.security.SecurityUtil.execute(SecurityUt
il.java:320)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu
rityUtil.java:175)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu
rityUtil.java:124)
at org.apache.catalina.core.StandardWrapper.initServlet(Standar
dWrapper.java:1270)
at org.apache.catalina.core.StandardWrapper.loadServlet(Standar
dWrapper.java:1195)
at org.apache.catalina.core.StandardWrapper.load(StandardWrappe
r.java:1085)
at org.apache.catalina.core.StandardContext.loadOnStartup(Stand
ardContext.java:5318)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5610)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.j
ava:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:899)
at org.apache.catalina.core.ContainerBase.access$000(ContainerB
ase.java:133)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru
n(ContainerBase.java:156)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru
n(ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBas
e.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.
java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(Host
Config.java:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(
HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(Executor
s.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[03/May/2017:21:22:02][localhost-startStop-1]: CMSEngine.shutdown()
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,uMGMT.CROSSCHX.COM IPA CA CT,C,C
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u
IPA13.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,uMGMT.CROSSCHX.COM IPA CA CT,C,C
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u
IPA12.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,uMGMT.CROSSCHX.COM IPA CA C,,
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u
=================================================
IPA11.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:52:43 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 15:56:20 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa11.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:52:46 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
==================================
IPA13.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 14:34:20 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 14:37:54 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa13.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 14:34:23 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
===========================
IPA12.MGMT
(root)>getcert list
Number of certificates and requests being tracked: 8.
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:14:51 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MGMT-CROSSCHX-COM
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Audit,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:29 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:26 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=CA Subsystem,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:00:28 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
expires: 2036-11-22 13:00:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-19 15:18:16 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=ipa12.mgmt.crosschx.com,O=MGMT.CROSSCHX.COM
expires: 2018-12-30 15:14:54 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=MGMT.CROSSCHX.COM
subject: CN=IPA RA,O=MGMT.CROSSCHX.COM
expires: 2018-11-12 13:01:34 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com

[Rob Crittenden]

Post by Michael Plemmons
I realized that I was not very clear in my statement about testing with
ldapsearch. I had initially run it without logging in with a DN. I was
just running the local ldapsearch -x command. I then tested on
ipa12.mgmt and ipa11.mgmt logging in with a full DN for the admin and
"cn=Directory Manager" from ipa12.mgmt (broken server) and ipa11.mgmt
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt and ipa11.mgmt as a non root user.
I also ran the command showing a line count for the output and the line
counts for each were the same when run from ipa12.mgmt and ipa11.mgmt.
ldapsearch -LLL -h ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com> -D "DN" -w PASSWORD -b
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
ldapsearch -LLL -h ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com> -D "cn=directory manager" -w PASSWORD dn

The CA has its own suffix and replication agreements. Given the auth
error and recent (5 months) renewal of CA credentials I'd check that the
CA agent authentication entries are correct.

Against each master with a CA run:

$ ldapsearch -LLL -x -D 'cn=directory manager' -W -b
uid=ipara,ou=people,o=ipaca description

The format is 2;serial#,subject,issuer

Then on each run:

# certutil -L -d /etc/httpd/alias -n ipaCert |grep Serial

The serial # should match that in the description everywhere.

rob

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and ipa12 do not
have agreements between each other.
It appears that either ipa12.mgmt lost some level of its replication
agreement with ipa13. I saw some level because users / hosts were
replicated between all systems but we started seeing DNS was not
resolving properly from ipa12. I do not know when this started.
When looking at replication agreements on ipa12 I did not see any
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show has master.
When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt is a replica.
When I run ipa-replica-manage ipa12.mgmt nothing returned.
I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com> on ipa12.mgmt
I then ran the following
ipa-replica-manage force-sync --from ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
ipa-replica-manage re-initialize --from ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
I was still seeing bad DNS returns when dig'ing against ipa12.mgmt.
I was able to create user and DNS records and see the information
replicated properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl start on
ipa12.mgmt because I wanted to make sure everything was running
fresh after the changes above. While IPA was staring up (DNS
started) we were able to see valid DNS queries returned but
pki-tomcat would not start.
I am not sure what I need to do in order to get this working. I
have included the output of certutil and getcert below from all
three servers as well as the debug output for pki.
While the IPA system is coming up I am able to successfully run
ldapsearch -x as the root user and see results. I am also able to
login with the "cn=Directory Manager" account and see results.
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: ===== DEBUG
SUBSYSTEM INITIALIZED =======
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to
look for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=debug
initialized debug
initSubsystem id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to
init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to
look for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=log
initialized log
initSubsystem id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to
init id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart at
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to
look for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=jss
initialized jss
initSubsystem id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to
init id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: DBSubsystem: init()
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
[03/May/2017:21:22:01][localhost-startStop-1]: LdapBoundConnFactory: init
LdapBoundConnFactory:doCloning true
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init()
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init begins
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init ends
[03/May/2017:21:22:01][localhost-startStop-1]: init: before
makeConnection errorIfDown is true
errorIfDown true
subsystemCert cert-pki-ca
[03/May/2017:21:22:02][localhost-startStop-1]: LdapJssSSLSocket: set
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com> port 636 Error
netscape.ldap.LDAPException: Authentication failed (48)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:166)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:130)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)
at
com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1169)
at
com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at
com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)
at
org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)
at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
at
org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at
org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)
at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect to LDAP
server host ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676)
at
com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1169)
at
com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1075)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at
com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)
at
org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)
at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
at
org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at
org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)
at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[03/May/2017:21:22:02][localhost-startStop-1]: CMSEngine.shutdown()
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI Server-Cert
u,u,u MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM> IPA CA CT,C,C
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/ Certificate
Nickname Trust Attributes SSL,S/MIME,JAR/XPI caSigningCert
cert-pki-ca CTu,Cu,Cu auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u IPA13.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname Trust
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C (root)>certutil -L -d
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert cert-pki-ca
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert cert-pki-ca u,u,u
IPA12.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname Trust
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> IPA CA C,, (root)>certutil -L -d
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert cert-pki-ca
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert cert-pki-ca u,u,u
================================================= IPA11.MGMT
(root)>getcert list Number of certificates and requests being
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa11.mgmt.crosschx.com
<http://ipa11.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:43 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
2018-11-12 13:00:29 UTC key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=CA Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=Certificate Authority,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25 UTC key
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign pre-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS
Certificate DB' CA: dogtag-ipa-renew-agent issuer: CN=Certificate
CN=ipa11.mgmt.crosschx.com
<http://ipa11.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:56:20 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa11.mgmt.crosschx.com
<http://ipa11.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:46 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_httpd track: yes
auto-renew: yes Request ID '20161229160009': status: MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes
================================== IPA13.MGMT (root)>getcert list
Number of certificates and requests being tracked: 8. Request ID
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:20 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
2018-11-12 13:00:29 UTC key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=CA Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=Certificate Authority,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25 UTC key
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign pre-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS
Certificate DB' CA: dogtag-ipa-renew-agent issuer: CN=Certificate
CN=ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 14:37:54 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:23 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_httpd track: yes
auto-renew: yes Request ID '20161229144146': status: MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes
=========================== IPA12.MGMT (root)>getcert list Number of
certificates and requests being tracked: 8. Request ID
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:51 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
2018-11-12 13:00:29 UTC key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=CA Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=Certificate Authority,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25 UTC key
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign pre-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS
Certificate DB' CA: dogtag-ipa-renew-agent issuer: CN=Certificate
CN=ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:18:16 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:54 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_httpd track: yes
auto-renew: yes Request ID '20161229152204': status: MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>

[Michael Plemmons]

I just realized that I sent the reply directly to Rob and not to the list.
My response is inline




*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons <

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com

Post by Rob Crittenden

Post by Michael Plemmons
I realized that I was not very clear in my statement about testing with
ldapsearch. I had initially run it without logging in with a DN. I was
just running the local ldapsearch -x command. I then tested on
ipa12.mgmt and ipa11.mgmt logging in with a full DN for the admin and
"cn=Directory Manager" from ipa12.mgmt (broken server) and ipa11.mgmt
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt and ipa11.mgmt as a non root user.
I also ran the command showing a line count for the output and the line
counts for each were the same when run from ipa12.mgmt and ipa11.mgmt.
ldapsearch -LLL -h ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com> -D "DN" -w PASSWORD -b
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
ldapsearch -LLL -h ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com> -D "cn=directory manager" -w PASSWORD

dn
The CA has its own suffix and replication agreements. Given the auth
error and recent (5 months) renewal of CA credentials I'd check that the
CA agent authentication entries are correct.
$ ldapsearch -LLL -x -D 'cn=directory manager' -W -b
uid=ipara,ou=people,o=ipaca description
The format is 2;serial#,subject,issuer
# certutil -L -d /etc/httpd/alias -n ipaCert |grep Serial
The serial # should match that in the description everywhere.
rob

On the CA (IPA13.MGMT) I ran the ldapsearch command and see that the
serial number is 7. I then ran the certutil command on all three servers
and the serial number is 7 as well.
I also ran the ldapsearch command against the other two servers and they
also showed a serial number of 7.

Post by Rob Crittenden

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and ipa12 do not
have agreements between each other.
It appears that either ipa12.mgmt lost some level of its replication
agreement with ipa13. I saw some level because users / hosts were
replicated between all systems but we started seeing DNS was not
resolving properly from ipa12. I do not know when this started.
When looking at replication agreements on ipa12 I did not see any
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show has master.
When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt is a

replica.

Post by Michael Plemmons
When I run ipa-replica-manage ipa12.mgmt nothing returned.
I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com> on

ipa12.mgmt

Post by Michael Plemmons
I then ran the following
ipa-replica-manage force-sync --from ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
ipa-replica-manage re-initialize --from ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
I was still seeing bad DNS returns when dig'ing against ipa12.mgmt.
I was able to create user and DNS records and see the information
replicated properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl start on
ipa12.mgmt because I wanted to make sure everything was running
fresh after the changes above. While IPA was staring up (DNS
started) we were able to see valid DNS queries returned but
pki-tomcat would not start.
I am not sure what I need to do in order to get this working. I
have included the output of certutil and getcert below from all
three servers as well as the debug output for pki.
While the IPA system is coming up I am able to successfully run
ldapsearch -x as the root user and see results. I am also able to
login with the "cn=Directory Manager" account and see results.
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: ===== DEBUG
SUBSYSTEM INITIALIZED =======
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart

at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to
look for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=debug
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized debug
initSubsystem id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to
init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/

ca_audit)

Post by Michael Plemmons
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart

at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to
look for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized log
initSubsystem id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to
init id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: restart

at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: about to
look for cert for auto-shutdown support:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: found
cert:auditSigningCert cert-pki-ca
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: done init id=jss
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: initialized jss
initSubsystem id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: CMSEngine: ready to
init id=dbs
[03/May/2017:21:22:01][localhost-startStop-1]: DBSubsystem: init()
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
init
LdapBoundConnFactory:doCloning true
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init()
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init

begins

Post by Michael Plemmons
[03/May/2017:21:22:01][localhost-startStop-1]: LdapAuthInfo: init

ends

Post by Michael Plemmons
[03/May/2017:21:22:01][localhost-startStop-1]: init: before
makeConnection errorIfDown is true
errorIfDown true
subsystemCert cert-pki-ca

set

Post by Michael Plemmons
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL handshake

happened

Post by Michael Plemmons
Could not connect to LDAP server host ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com> port 636 Error
netscape.ldap.LDAPException: Authentication failed (48)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.

makeConnection(LdapBoundConnFactory.java:205)

Post by Michael Plemmons
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.

init(LdapBoundConnFactory.java:166)

Post by Michael Plemmons
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.

init(LdapBoundConnFactory.java:130)
654)

Post by Michael Plemmons
at
com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine

.java:1169)

Post by Michael Plemmons
at
com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngin

e.java:1075)

Post by Michael Plemmons
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at
com.netscape.cms.servlet.base.CMSStartServlet.init(CMSS

tartServlet.java:114)

Post by Michael Plemmons
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcc

essorImpl.java:62)

Post by Michael Plemmons
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingM

ethodAccessorImpl.java:43)

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUti

l.java:288)

Post by Michael Plemmons
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUti

l.java:285)

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at
org.apache.catalina.security.SecurityUtil.execute(SecurityU

til.java:320)

Post by Michael Plemmons
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(Sec

urityUtil.java:175)

Post by Michael Plemmons
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(Sec

urityUtil.java:124)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardWrapper.initServlet(Standa

rdWrapper.java:1270)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardWrapper.loadServlet(Standa

rdWrapper.java:1195)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardWrapper.load(StandardWrapp

er.java:1085)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardContext.loadOnStartup(Stan

dardContext.java:5318)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardContext.startInternal(Stan

dardContext.java:5610)

Post by Michael Plemmons
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.

java:147)

Post by Michael Plemmons
at
org.apache.catalina.core.ContainerBase.addChildInternal(Con

tainerBase.java:899)

Post by Michael Plemmons
at
org.apache.catalina.core.ContainerBase.access$000(Container

Base.java:133)

Post by Michael Plemmons
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.

run(ContainerBase.java:156)

Post by Michael Plemmons
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.

run(ContainerBase.java:145)

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native Method)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBa

se.java:873)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardHost.addChild(StandardHost

.java:652)

Post by Michael Plemmons
at
org.apache.catalina.startup.HostConfig.deployDescriptor(Hos

tConfig.java:679)

Post by Michael Plemmons
at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run

(HostConfig.java:1966)

Post by Michael Plemmons
at
java.util.concurrent.Executors$RunnableAdapter.call(

Executors.java:511)

Post by Michael Plemmons
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoo

lExecutor.java:1142)

Post by Michael Plemmons
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPo

olExecutor.java:617)

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect to LDAP
server host ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com
port 636 Error netscape.ldap.LDAPException: Authentication failed

(48)
676)

Post by Michael Plemmons
at
com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine

.java:1169)

Post by Michael Plemmons
at
com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngin

e.java:1075)

Post by Michael Plemmons
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at
com.netscape.cms.servlet.base.CMSStartServlet.init(CMSS

tartServlet.java:114)

Post by Michael Plemmons
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcc

essorImpl.java:62)

Post by Michael Plemmons
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingM

ethodAccessorImpl.java:43)

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUti

l.java:288)

Post by Michael Plemmons
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUti

l.java:285)

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at
org.apache.catalina.security.SecurityUtil.execute(SecurityU

til.java:320)

Post by Michael Plemmons
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(Sec

urityUtil.java:175)

Post by Michael Plemmons
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(Sec

urityUtil.java:124)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardWrapper.initServlet(Standa

rdWrapper.java:1270)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardWrapper.loadServlet(Standa

rdWrapper.java:1195)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardWrapper.load(StandardWrapp

er.java:1085)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardContext.loadOnStartup(Stan

dardContext.java:5318)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardContext.startInternal(Stan

dardContext.java:5610)

Post by Michael Plemmons
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.

java:147)

Post by Michael Plemmons
at
org.apache.catalina.core.ContainerBase.addChildInternal(Con

tainerBase.java:899)

Post by Michael Plemmons
at
org.apache.catalina.core.ContainerBase.access$000(Container

Base.java:133)

Post by Michael Plemmons
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.

run(ContainerBase.java:156)

Post by Michael Plemmons
at
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.

run(ContainerBase.java:145)

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native Method)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBa

se.java:873)

Post by Michael Plemmons
at
org.apache.catalina.core.StandardHost.addChild(StandardHost

.java:652)

Post by Michael Plemmons
at
org.apache.catalina.startup.HostConfig.deployDescriptor(Hos

tConfig.java:679)

Post by Michael Plemmons
at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run

(HostConfig.java:1966)

Post by Michael Plemmons
at
java.util.concurrent.Executors$RunnableAdapter.call(

Executors.java:511)

Post by Michael Plemmons
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoo

lExecutor.java:1142)

Post by Michael Plemmons
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPo

olExecutor.java:617)

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)
[03/May/2017:21:22:02][localhost-startStop-1]: CMSEngine.shutdown()
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI Server-Cert
u,u,u MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM> IPA CA CT,C,C
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/ Certificate
Nickname Trust Attributes SSL,S/MIME,JAR/XPI caSigningCert
cert-pki-ca CTu,Cu,Cu auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u subsystemCert cert-pki-ca u,u,u
Server-Cert cert-pki-ca u,u,u IPA13.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname Trust
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C (root)>certutil -L -d
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust

Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert cert-pki-ca
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert cert-pki-ca u,u,u
IPA12.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname Trust
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> IPA CA C,, (root)>certutil -L -d
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust

Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert cert-pki-ca
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert cert-pki-ca u,u,u
================================================= IPA11.MGMT
(root)>getcert list Number of certificates and requests being
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa11.mgmt.crosschx.com
<http://ipa11.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:43 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='auditSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='auditSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
2018-11-12 13:00:29 UTC key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='ocspSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='ocspSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='subsystemCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='subsystemCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=CA Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='caSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='caSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=Certificate Authority,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25 UTC key
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign pre-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert

cert-pki-ca',token='NSS

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert

cert-pki-ca',token='NSS

Post by Michael Plemmons
Certificate DB' CA: dogtag-ipa-renew-agent issuer: CN=Certificate
CN=ipa11.mgmt.crosschx.com
<http://ipa11.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:56:20 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cer

t',token='NSS

Post by Michael Plemmons
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cer

t',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa11.mgmt.crosschx.com
<http://ipa11.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:46 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_httpd track: yes
auto-renew: yes Request ID '20161229160009': status: MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons
Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

yes

Post by Michael Plemmons
================================== IPA13.MGMT (root)>getcert list
Number of certificates and requests being tracked: 8. Request ID
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:20 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='auditSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='auditSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
2018-11-12 13:00:29 UTC key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='ocspSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='ocspSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='subsystemCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='subsystemCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=CA Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='caSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='caSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=Certificate Authority,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25 UTC key
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign pre-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert

cert-pki-ca',token='NSS

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert

cert-pki-ca',token='NSS

Post by Michael Plemmons
Certificate DB' CA: dogtag-ipa-renew-agent issuer: CN=Certificate
CN=ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 14:37:54 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cer

t',token='NSS

Post by Michael Plemmons
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cer

t',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:23 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_httpd track: yes
auto-renew: yes Request ID '20161229144146': status: MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons
Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

yes

Post by Michael Plemmons
=========================== IPA12.MGMT (root)>getcert list Number of
certificates and requests being tracked: 8. Request ID
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:51 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='auditSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='auditSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
2018-11-12 13:00:29 UTC key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='ocspSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='ocspSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='subsystemCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='subsystemCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=CA Subsystem,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='caSigningCert

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',

nickname='caSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
CN=Certificate Authority,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25 UTC key
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign pre-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert

cert-pki-ca',token='NSS

Post by Michael Plemmons
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert

cert-pki-ca',token='NSS

Post by Michael Plemmons
Certificate DB' CA: dogtag-ipa-renew-agent issuer: CN=Certificate
CN=ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:18:16 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cer

t',token='NSS

Post by Michael Plemmons
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cer

t',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
CN=ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:54 UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
command: /usr/libexec/ipa/certmonger/restart_httpd track: yes
auto-renew: yes Request ID '20161229152204': status: MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons
Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

yes

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>

[Rob Crittenden]

Post by Michael Plemmons
I just realized that I sent the reply directly to Rob and not to the
list. My response is inline

Ok, this is actually good news.

I made a similar proposal in another case and I was completely wrong.
Flo had the user do something and it totally fixed their auth error, I
just can't remember what it was or find the e-mail thread. I'm pretty
sure it was this calendar year though.

rob

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>

Post by Michael Plemmons
I realized that I was not very clear in my statement about

testing with

Post by Michael Plemmons
ldapsearch. I had initially run it without logging in with a

DN. I was

Post by Michael Plemmons
just running the local ldapsearch -x command. I then tested on
ipa12.mgmt and ipa11.mgmt logging in with a full DN for the

admin and

Post by Michael Plemmons
"cn=Directory Manager" from ipa12.mgmt (broken server) and

ipa11.mgmt

Post by Michael Plemmons
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt and ipa11.mgmt as a non

root user.

Post by Michael Plemmons
I also ran the command showing a line count for the output and

the line

Post by Michael Plemmons
counts for each were the same when run from ipa12.mgmt and

ipa11.mgmt.

Post by Michael Plemmons
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "DN" -w PASSWORD -b

Post by Michael Plemmons
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "cn=directory manager" -w
PASSWORD dn
The CA has its own suffix and replication agreements. Given the auth
error and recent (5 months) renewal of CA credentials I'd check that the
CA agent authentication entries are correct.
$ ldapsearch -LLL -x -D 'cn=directory manager' -W -b
uid=ipara,ou=people,o=ipaca description
The format is 2;serial#,subject,issuer
# certutil -L -d /etc/httpd/alias -n ipaCert |grep Serial
The serial # should match that in the description everywhere.
rob
On the CA (IPA13.MGMT) I ran the ldapsearch command and see that the
serial number is 7. I then ran the certutil command on all three
servers and the serial number is 7 as well.
I also ran the ldapsearch command against the other two servers and
they also showed a serial number of 7.

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

Post by Michael Plemmons
On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and

ipa12 do not

Post by Michael Plemmons
have agreements between each other.
It appears that either ipa12.mgmt lost some level of its

replication

Post by Michael Plemmons
agreement with ipa13. I saw some level because users /

hosts were

Post by Michael Plemmons
replicated between all systems but we started seeing DNS

was not

Post by Michael Plemmons
resolving properly from ipa12. I do not know when this

started.

Post by Michael Plemmons
When looking at replication agreements on ipa12 I did not

see any

Post by Michael Plemmons
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show

has master.

Post by Michael Plemmons
When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt

is a replica.

Post by Michael Plemmons
When I run ipa-replica-manage ipa12.mgmt nothing returned.
I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>

<http://ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>> on ipa12.mgmt

Post by Michael Plemmons
I then ran the following
ipa-replica-manage force-sync --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
ipa-replica-manage re-initialize --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
I was still seeing bad DNS returns when dig'ing against

ipa12.mgmt.

Post by Michael Plemmons
I was able to create user and DNS records and see the

information

Post by Michael Plemmons
replicated properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl start on
ipa12.mgmt because I wanted to make sure everything was

running

Post by Michael Plemmons
fresh after the changes above. While IPA was staring up (DNS
started) we were able to see valid DNS queries returned but
pki-tomcat would not start.
I am not sure what I need to do in order to get this

working. I

Post by Michael Plemmons
have included the output of certutil and getcert below

from all

Post by Michael Plemmons
three servers as well as the debug output for pki.
While the IPA system is coming up I am able to

successfully run

Post by Michael Plemmons
ldapsearch -x as the root user and see results. I am also

able to

Post by Michael Plemmons
login with the "cn=Directory Manager" account and see results.
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: ===== DEBUG
SUBSYSTEM INITIALIZED =======
============================================

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=debug
initialized debug
initSubsystem id=log

ready to

Post by Michael Plemmons
init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating

RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)

Post by Michael Plemmons
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=log
initialized log
initSubsystem id=jss

ready to

Post by Michael Plemmons
init id=jss

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=jss
initialized jss
initSubsystem id=dbs

ready to

Post by Michael Plemmons
init id=dbs

DBSubsystem: init()

Post by Michael Plemmons
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
init
LdapBoundConnFactory:doCloning true

LdapAuthInfo: init()
LdapAuthInfo: init begins
LdapAuthInfo: init ends

Post by Michael Plemmons
[03/May/2017:21:22:01][localhost-startStop-1]: init: before
makeConnection errorIfDown is true
errorIfDown true
SSLClientCertificateSelectionCB: Setting desired cert
subsystemCert cert-pki-ca

LdapJssSSLSocket: set

Post by Michael Plemmons
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL

handshake happened

Post by Michael Plemmons
Could not connect to LDAP server host

ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> port 636 Error

Post by Michael Plemmons
netscape.ldap.LDAPException: Authentication failed (48)
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:166)

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:130)

Post by Michael Plemmons
at

com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1169)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1075)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(Subject.java:549)

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

Post by Michael Plemmons
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect to LDAP
server host ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com> <http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
port 636 Error netscape.ldap.LDAPException: Authentication

failed (48)

Post by Michael Plemmons
at

com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1169)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1075)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(Subject.java:549)

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

Post by Michael Plemmons
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)

CMSEngine.shutdown()

Post by Michael Plemmons
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI

Server-Cert

Post by Michael Plemmons
u,u,u MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C

Post by Michael Plemmons
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/

Certificate

Post by Michael Plemmons
Nickname Trust Attributes SSL,S/MIME,JAR/XPI caSigningCert
cert-pki-ca CTu,Cu,Cu auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u subsystemCert

cert-pki-ca u,u,u

Post by Michael Plemmons
Server-Cert cert-pki-ca u,u,u IPA13.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C (root)>certutil -L -d
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust

Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
IPA12.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA C,, (root)>certutil -L -d
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust

Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
================================================= IPA11.MGMT
(root)>getcert list Number of certificates and requests being

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <http://ipa11.mgmt.crosschx.com>
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:43

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229155652': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign

pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <http://ipa11.mgmt.crosschx.com>
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:56:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <http://ipa11.mgmt.crosschx.com>
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:46

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd track: yes

MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
================================== IPA13.MGMT

(root)>getcert list

Post by Michael Plemmons
Number of certificates and requests being tracked: 8.

Request ID

Post by Michael Plemmons
'20161229143449': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229143826': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign

pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 14:37:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:23

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd track: yes

MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
=========================== IPA12.MGMT (root)>getcert list

Number of

Post by Michael Plemmons
certificates and requests being tracked: 8. Request ID
'20161229151518': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.txt'

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:51

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229151850': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign

pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:18:16

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd track: yes

MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

[Michael Plemmons]

*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

Post by Rob Crittenden

Post by Michael Plemmons
I just realized that I sent the reply directly to Rob and not to the
list. My response is inline

Ok, this is actually good news.
I made a similar proposal in another case and I was completely wrong.
Flo had the user do something and it totally fixed their auth error, I
just can't remember what it was or find the e-mail thread. I'm pretty
sure it was this calendar year though.
rob

Do you or Flo know what I could search for in the past emails to find the
answer to the problem?

Post by Rob Crittenden

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>

Post by Michael Plemmons
I realized that I was not very clear in my statement about

testing with

Post by Michael Plemmons
ldapsearch. I had initially run it without logging in with a

DN. I was

Post by Michael Plemmons
just running the local ldapsearch -x command. I then tested on
ipa12.mgmt and ipa11.mgmt logging in with a full DN for the

admin and

Post by Michael Plemmons
"cn=Directory Manager" from ipa12.mgmt (broken server) and

ipa11.mgmt

Post by Michael Plemmons
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt and ipa11.mgmt as a non

root user.

Post by Michael Plemmons
I also ran the command showing a line count for the output and

the line

Post by Michael Plemmons
counts for each were the same when run from ipa12.mgmt and

ipa11.mgmt.

Post by Michael Plemmons
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "DN" -w PASSWORD -b

Post by Michael Plemmons
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "cn=directory manager" -w
PASSWORD dn
The CA has its own suffix and replication agreements. Given the

auth

Post by Michael Plemmons
error and recent (5 months) renewal of CA credentials I'd check
that the
CA agent authentication entries are correct.
$ ldapsearch -LLL -x -D 'cn=directory manager' -W -b
uid=ipara,ou=people,o=ipaca description
The format is 2;serial#,subject,issuer
# certutil -L -d /etc/httpd/alias -n ipaCert |grep Serial
The serial # should match that in the description everywhere.
rob
On the CA (IPA13.MGMT) I ran the ldapsearch command and see that the
serial number is 7. I then ran the certutil command on all three
servers and the serial number is 7 as well.
I also ran the ldapsearch command against the other two servers and
they also showed a serial number of 7.

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

Post by Michael Plemmons
On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and

ipa12 do not

Post by Michael Plemmons
have agreements between each other.
It appears that either ipa12.mgmt lost some level of its

replication

Post by Michael Plemmons
agreement with ipa13. I saw some level because users /

hosts were

Post by Michael Plemmons
replicated between all systems but we started seeing DNS

was not

Post by Michael Plemmons
resolving properly from ipa12. I do not know when this

started.

Post by Michael Plemmons
When looking at replication agreements on ipa12 I did not

see any

Post by Michael Plemmons
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show

has master.

Post by Michael Plemmons
When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt

is a replica.

Post by Michael Plemmons
When I run ipa-replica-manage ipa12.mgmt nothing returned.
I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>

<http://ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com

Post by Michael Plemmons
ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>> on ipa12.mgmt

Post by Michael Plemmons
I then ran the following
ipa-replica-manage force-sync --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
ipa-replica-manage re-initialize --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
I was still seeing bad DNS returns when dig'ing against

ipa12.mgmt.

Post by Michael Plemmons
I was able to create user and DNS records and see the

information

Post by Michael Plemmons
replicated properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl start

on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt because I wanted to make sure everything was

running

Post by Michael Plemmons
fresh after the changes above. While IPA was staring up

(DNS

Post by Michael Plemmons

Post by Michael Plemmons
started) we were able to see valid DNS queries returned but
pki-tomcat would not start.
I am not sure what I need to do in order to get this

working. I

Post by Michael Plemmons
have included the output of certutil and getcert below

from all

Post by Michael Plemmons
three servers as well as the debug output for pki.
While the IPA system is coming up I am able to

successfully run

Post by Michael Plemmons
ldapsearch -x as the root user and see results. I am also

able to

Post by Michael Plemmons
login with the "cn=Directory Manager" account and see

results.

Post by Michael Plemmons

Post by Michael Plemmons
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: =====

DEBUG

Post by Michael Plemmons

Post by Michael Plemmons
SUBSYSTEM INITIALIZED =======
============================================

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=debug
initialized debug
initSubsystem id=log

ready to

Post by Michael Plemmons
init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating

RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/

signedAudit/ca_audit)

Post by Michael Plemmons

Post by Michael Plemmons
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/

transactions)

Post by Michael Plemmons
restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=log
initialized log
initSubsystem id=jss

ready to

Post by Michael Plemmons
init id=jss

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=jss
initialized jss
initSubsystem id=dbs

ready to

Post by Michael Plemmons
init id=dbs

DBSubsystem: init()

Post by Michael Plemmons
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
init
LdapBoundConnFactory:doCloning true

LdapAuthInfo: init()
LdapAuthInfo: init begins
LdapAuthInfo: init ends

before

Post by Michael Plemmons

Post by Michael Plemmons
makeConnection errorIfDown is true
errorIfDown true
SSLClientCertificateSelectionCB: Setting desired cert
subsystemCert cert-pki-ca

LdapJssSSLSocket: set

Post by Michael Plemmons
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL

handshake happened

Post by Michael Plemmons
Could not connect to LDAP server host

ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> port 636 Error

Post by Michael Plemmons
netscape.ldap.LDAPException: Authentication failed (48)
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.

makeConnection(LdapBoundConnFactory.java:205)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(

LdapBoundConnFactory.java:166)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(

LdapBoundConnFactory.java:130)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(

CMSEngine.java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(

CMSEngine.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(

CMSStartServlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(

NativeMethodAccessorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(

DelegatingMethodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(

SecurityUtil.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(

SecurityUtil.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(

Subject.java:549)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(

SecurityUtil.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(

SecurityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(

SecurityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(

StandardWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(

StandardWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(

StandardWrapper.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(

StandardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(

StandardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(

LifecycleBase.java:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(

ContainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(

ContainerBase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(

ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(

ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(

ContainerBase.java:873)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(

StandardHost.java:652)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(

HostConfig.java:679)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.

run(HostConfig.java:1966)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.

call(Executors.java:511)

Post by Michael Plemmons

Post by Michael Plemmons
at java.util.concurrent.FutureTask.run(FutureTask.

java:266)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor.runWorker(

ThreadPoolExecutor.java:1142)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(

ThreadPoolExecutor.java:617)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect to

LDAP

Post by Michael Plemmons

Post by Michael Plemmons
server host ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com> <http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
port 636 Error netscape.ldap.LDAPException: Authentication

failed (48)

Post by Michael Plemmons
at

com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(

CMSEngine.java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(

CMSEngine.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(

CMSStartServlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(

NativeMethodAccessorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(

DelegatingMethodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(

SecurityUtil.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(

SecurityUtil.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(

Subject.java:549)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(

SecurityUtil.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(

SecurityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(

SecurityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(

StandardWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(

StandardWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(

StandardWrapper.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(

StandardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(

StandardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(

LifecycleBase.java:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(

ContainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(

ContainerBase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(

ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(

ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(

ContainerBase.java:873)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(

StandardHost.java:652)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(

HostConfig.java:679)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.

run(HostConfig.java:1966)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.

call(Executors.java:511)

Post by Michael Plemmons

Post by Michael Plemmons
at java.util.concurrent.FutureTask.run(FutureTask.

java:266)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor.runWorker(

ThreadPoolExecutor.java:1142)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(

ThreadPoolExecutor.java:617)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)

CMSEngine.shutdown()

Post by Michael Plemmons
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCHX-COM/
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI

Server-Cert

Post by Michael Plemmons
u,u,u MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C

Post by Michael Plemmons
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/

Certificate

Post by Michael Plemmons
Nickname Trust Attributes SSL,S/MIME,JAR/XPI caSigningCert
cert-pki-ca CTu,Cu,Cu auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u subsystemCert

cert-pki-ca u,u,u

Post by Michael Plemmons
Server-Cert cert-pki-ca u,u,u IPA13.MGMT (root)>certutil

-L -d

Post by Michael Plemmons

Post by Michael Plemmons
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C (root)>certutil

-L -d

Post by Michael Plemmons

Post by Michael Plemmons
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust

Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
IPA12.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA C,, (root)>certutil -L

-d

Post by Michael Plemmons

Post by Michael Plemmons
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust

Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
=================================================

IPA11.MGMT

Post by Michael Plemmons

Post by Michael Plemmons
(root)>getcert list Number of certificates and requests

being

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-

COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/

pwdfile.txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-

COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <http://ipa11.mgmt.crosschx.com
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:43

UTC key
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229155652': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

auditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

auditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

subsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

subsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign

pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

Server-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

Server-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <http://ipa11.mgmt.crosschx.com
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:56:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-

emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-

Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-

Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <http://ipa11.mgmt.crosschx.com
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:46

UTC key
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons
post-save

yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='

ipaCert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='

ipaCert',token='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
================================== IPA13.MGMT

(root)>getcert list

Post by Michael Plemmons
Number of certificates and requests being tracked: 8.

Request ID

Post by Michael Plemmons
'20161229143449': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-

COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/

pwdfile.txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-

COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229143826': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

auditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

auditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

subsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

subsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign

pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

Server-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

Server-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 14:37:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-

emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-

Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-

Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:23

UTC key
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons
post-save

yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='

ipaCert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='

ipaCert',token='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
=========================== IPA12.MGMT (root)>getcert list

Number of

Post by Michael Plemmons
certificates and requests being tracked: 8. Request ID
'20161229151518': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-

COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/

pwdfile.txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-

COM',nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:51

UTC key
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229151850': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

auditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

auditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

subsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

subsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign

pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

Server-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='

Server-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:18:16

UTC key
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-

emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-

Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-

Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons
post-save

yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='

ipaCert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='

ipaCert',token='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,

dataEncipherment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

[Michael Plemmons]

I think I found the email thread. Asking for help with crashed freeIPA
istance. That email pointed to this link,
https://www.redhat.com/archives/freeipa-users/2017-January/msg00215.html.
That link talked about changing the CS.cfg file to use port 389 for PKI to
auth to LDAP. I made the necessary changes and PKI came up successfully.




*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

On Fri, May 5, 2017 at 3:19 PM, Michael Plemmons <

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com

Post by Rob Crittenden

Post by Michael Plemmons
I just realized that I sent the reply directly to Rob and not to the
list. My response is inline

Ok, this is actually good news.
I made a similar proposal in another case and I was completely wrong.
Flo had the user do something and it totally fixed their auth error, I
just can't remember what it was or find the e-mail thread. I'm pretty
sure it was this calendar year though.
rob

Do you or Flo know what I could search for in the past emails to find the
answer to the problem?

Post by Rob Crittenden

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>

Post by Michael Plemmons
I realized that I was not very clear in my statement about

testing with

Post by Michael Plemmons
ldapsearch. I had initially run it without logging in with a

DN. I was

Post by Michael Plemmons
just running the local ldapsearch -x command. I then tested

on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt and ipa11.mgmt logging in with a full DN for the

admin and

Post by Michael Plemmons
"cn=Directory Manager" from ipa12.mgmt (broken server) and

ipa11.mgmt

Post by Michael Plemmons
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt and ipa11.mgmt as a non

root user.

Post by Michael Plemmons
I also ran the command showing a line count for the output and

the line

Post by Michael Plemmons
counts for each were the same when run from ipa12.mgmt and

ipa11.mgmt.

Post by Michael Plemmons
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "DN" -w PASSWORD -b

Post by Michael Plemmons
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "cn=directory manager" -w
PASSWORD dn
The CA has its own suffix and replication agreements. Given the

auth

Post by Michael Plemmons
error and recent (5 months) renewal of CA credentials I'd check
that the
CA agent authentication entries are correct.
$ ldapsearch -LLL -x -D 'cn=directory manager' -W -b
uid=ipara,ou=people,o=ipaca description
The format is 2;serial#,subject,issuer
# certutil -L -d /etc/httpd/alias -n ipaCert |grep Serial
The serial # should match that in the description everywhere.
rob
On the CA (IPA13.MGMT) I ran the ldapsearch command and see that the
serial number is 7. I then ran the certutil command on all three
servers and the serial number is 7 as well.
I also ran the ldapsearch command against the other two servers and
they also showed a serial number of 7.

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

Post by Michael Plemmons
On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and

ipa12 do not

Post by Michael Plemmons
have agreements between each other.
It appears that either ipa12.mgmt lost some level of its

replication

Post by Michael Plemmons
agreement with ipa13. I saw some level because users /

hosts were

Post by Michael Plemmons
replicated between all systems but we started seeing DNS

was not

Post by Michael Plemmons
resolving properly from ipa12. I do not know when this

started.

Post by Michael Plemmons
When looking at replication agreements on ipa12 I did not

see any

Post by Michael Plemmons
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show

has master.

Post by Michael Plemmons
When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt

is a replica.

Post by Michael Plemmons
When I run ipa-replica-manage ipa12.mgmt nothing returned.
I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>

<http://ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com

Post by Michael Plemmons
ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>> on ipa12.mgmt

Post by Michael Plemmons
I then ran the following
ipa-replica-manage force-sync --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
ipa-replica-manage re-initialize --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
I was still seeing bad DNS returns when dig'ing against

ipa12.mgmt.

Post by Michael Plemmons
I was able to create user and DNS records and see the

information

Post by Michael Plemmons
replicated properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl

start on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt because I wanted to make sure everything was

running

Post by Michael Plemmons
fresh after the changes above. While IPA was staring up

(DNS

Post by Michael Plemmons

Post by Michael Plemmons
started) we were able to see valid DNS queries returned

but

Post by Michael Plemmons

Post by Michael Plemmons
pki-tomcat would not start.
I am not sure what I need to do in order to get this

working. I

Post by Michael Plemmons
have included the output of certutil and getcert below

from all

Post by Michael Plemmons
three servers as well as the debug output for pki.
While the IPA system is coming up I am able to

successfully run

Post by Michael Plemmons
ldapsearch -x as the root user and see results. I am also

able to

Post by Michael Plemmons
login with the "cn=Directory Manager" account and see

results.

Post by Michael Plemmons

Post by Michael Plemmons
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: =====

DEBUG

Post by Michael Plemmons

Post by Michael Plemmons
SUBSYSTEM INITIALIZED =======
============================================

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=debug
initialized debug
initSubsystem id=log

ready to

Post by Michael Plemmons
init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating

RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/

ca_audit)

Post by Michael Plemmons

Post by Michael Plemmons
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/p

ki-tomcat/logs/ca/transactions)

Post by Michael Plemmons
restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=log
initialized log
initSubsystem id=jss

ready to

Post by Michael Plemmons
init id=jss

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=jss
initialized jss
initSubsystem id=dbs

ready to

Post by Michael Plemmons
init id=dbs

DBSubsystem: init()

Post by Michael Plemmons
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
init
LdapBoundConnFactory:doCloning true

LdapAuthInfo: init()
LdapAuthInfo: init begins
LdapAuthInfo: init ends

before

Post by Michael Plemmons

Post by Michael Plemmons
makeConnection errorIfDown is true
errorIfDown true
SSLClientCertificateSelectionCB: Setting desired cert
subsystemCert cert-pki-ca

LdapJssSSLSocket: set

Post by Michael Plemmons
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL

handshake happened

Post by Michael Plemmons
Could not connect to LDAP server host

ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> port 636 Error

Post by Michael Plemmons
netscape.ldap.LDAPException: Authentication failed (48)
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConne

ction(LdapBoundConnFactory.java:205)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap

BoundConnFactory.java:166)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap

BoundConnFactory.java:130)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.

java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine

.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS

ervlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce

ssorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe

thodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(Subject

.java:549)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(SecurityUt

il.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(Standar

dWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(Standar

dWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(StandardWrappe

r.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(Stand

ardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(Stand

ardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(LifecycleBase.

java:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(Cont

ainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(ContainerB

ase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.

run(ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.

run(ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(ContainerBas

e.java:873)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(StandardHost.

java:652)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(Host

Config.java:679)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.run(

HostConfig.java:1966)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.call(

Executors.java:511)

Post by Michael Plemmons

Post by Michael Plemmons
at java.util.concurrent.FutureTas

k.run(FutureTask.java:266)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool

Executor.java:1142)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo

lExecutor.java:617)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect to

LDAP

Post by Michael Plemmons

Post by Michael Plemmons
server host ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com> <http://ipa12.mgmt.crosschx.co

m

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
port 636 Error netscape.ldap.LDAPException: Authentication

failed (48)

Post by Michael Plemmons
at

com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676)

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.

java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine

.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS

ervlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce

ssorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe

thodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(Subject

.java:549)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(SecurityUt

il.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(Standar

dWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(Standar

dWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(StandardWrappe

r.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(Stand

ardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(Stand

ardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(LifecycleBase.

java:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(Cont

ainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(ContainerB

ase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.

run(ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.

run(ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(ContainerBas

e.java:873)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(StandardHost.

java:652)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(Host

Config.java:679)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.run(

HostConfig.java:1966)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.call(

Executors.java:511)

Post by Michael Plemmons

Post by Michael Plemmons
at java.util.concurrent.FutureTas

k.run(FutureTask.java:266)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool

Executor.java:1142)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo

lExecutor.java:617)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)

CMSEngine.shutdown()

Post by Michael Plemmons
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCH

X-COM/

Post by Michael Plemmons

Post by Michael Plemmons
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI

Server-Cert

Post by Michael Plemmons
u,u,u MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C

Post by Michael Plemmons
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/

Certificate

Post by Michael Plemmons
Nickname Trust Attributes SSL,S/MIME,JAR/XPI caSigningCert
cert-pki-ca CTu,Cu,Cu auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u subsystemCert

cert-pki-ca u,u,u

Post by Michael Plemmons
Server-Cert cert-pki-ca u,u,u IPA13.MGMT (root)>certutil

-L -d

Post by Michael Plemmons

Post by Michael Plemmons
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C (root)>certutil

-L -d

Post by Michael Plemmons

Post by Michael Plemmons
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust

Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
IPA12.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA C,, (root)>certutil -L

-d

Post by Michael Plemmons

Post by Michael Plemmons
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname Trust

Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
=================================================

IPA11.MGMT

Post by Michael Plemmons

Post by Michael Plemmons
(root)>getcert list Number of certificates and requests

being

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.

txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <http://ipa11.mgmt.crosschx.co

m>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:43

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229155652': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM

digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiatio

n,keyCertSign,cRLSign

Post by Michael Plemmons
pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <http://ipa11.mgmt.crosschx.co

m>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:56:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientA

uth,id-kp-emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <http://ipa11.mgmt.crosschx.co

m>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:46

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd

track: yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
================================== IPA13.MGMT

(root)>getcert list

Post by Michael Plemmons
Number of certificates and requests being tracked: 8.

Request ID

Post by Michael Plemmons
'20161229143449': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.

txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.co

m>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229143826': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM

digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiatio

n,keyCertSign,cRLSign

Post by Michael Plemmons
pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.co

m>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 14:37:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientA

uth,id-kp-emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.co

m>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:23

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd

track: yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
=========================== IPA12.MGMT (root)>getcert list

Number of

Post by Michael Plemmons
certificates and requests being tracked: 8. Request ID
'20161229151518': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/pwdfile.

txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',

nickname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.co

m>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:51

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229151850': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM

digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiatio

n,keyCertSign,cRLSign

Post by Michael Plemmons
pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.co

m>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:18:16

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientA

uth,id-kp-emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.co

m>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd

track: yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',

token='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

[Michael Plemmons]

The PKI service came up successfully but only when it uses BasicAuth rather
than SSL auth. I am not sure about what I need to do in order to get the
auth working over SSL again.

None of the certs are expired when I run getcert list and ipa-getcert list.

Since the failure is with attempts to login to LDAP over 636. I have been
attempting to auth to LDAP via port 636 and the ldapsearch is not
completing. When looking at packet captures, I see some the TCP handshake
and what appears to be the start of a SSL process and then everything hangs.

What is the proper method to test performing a ldapsearch over 636? Also,
the CS.cfg shows it wants to auth as cn=Directory Manager. I can
successfully auth with cn=Directory Manager over 389 but I think I am not
performing ldapsearch over 636 correctly.




*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

On Fri, May 5, 2017 at 3:33 PM, Michael Plemmons <

Post by Michael Plemmons
I think I found the email thread. Asking for help with crashed freeIPA
istance. That email pointed to this link, https://www.redhat.com/a
rchives/freeipa-users/2017-January/msg00215.html. That link talked about
changing the CS.cfg file to use port 389 for PKI to auth to LDAP. I made
the necessary changes and PKI came up successfully.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com
On Fri, May 5, 2017 at 3:19 PM, Michael Plemmons <

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com

Post by Rob Crittenden

Post by Michael Plemmons
I just realized that I sent the reply directly to Rob and not to the
list. My response is inline

Ok, this is actually good news.
I made a similar proposal in another case and I was completely wrong.
Flo had the user do something and it totally fixed their auth error, I
just can't remember what it was or find the e-mail thread. I'm pretty
sure it was this calendar year though.
rob

Do you or Flo know what I could search for in the past emails to find the
answer to the problem?

Post by Rob Crittenden

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Thu, May 4, 2017 at 9:24 AM, Rob Crittenden <

Post by Michael Plemmons
I realized that I was not very clear in my statement about

testing with

Post by Michael Plemmons
ldapsearch. I had initially run it without logging in with a

DN. I was

Post by Michael Plemmons
just running the local ldapsearch -x command. I then tested

on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt and ipa11.mgmt logging in with a full DN for the

admin and

Post by Michael Plemmons
"cn=Directory Manager" from ipa12.mgmt (broken server) and

ipa11.mgmt

Post by Michael Plemmons
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt and ipa11.mgmt as a non

root user.

Post by Michael Plemmons
I also ran the command showing a line count for the output

and

Post by Michael Plemmons
the line

Post by Michael Plemmons
counts for each were the same when run from ipa12.mgmt and

ipa11.mgmt.

Post by Michael Plemmons
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "DN" -w PASSWORD -b

Post by Michael Plemmons
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "cn=directory manager" -w
PASSWORD dn
The CA has its own suffix and replication agreements. Given

the auth

Post by Michael Plemmons
error and recent (5 months) renewal of CA credentials I'd check
that the
CA agent authentication entries are correct.
$ ldapsearch -LLL -x -D 'cn=directory manager' -W -b
uid=ipara,ou=people,o=ipaca description
The format is 2;serial#,subject,issuer
# certutil -L -d /etc/httpd/alias -n ipaCert |grep Serial
The serial # should match that in the description everywhere.
rob
On the CA (IPA13.MGMT) I ran the ldapsearch command and see that

the

Post by Michael Plemmons
serial number is 7. I then ran the certutil command on all three
servers and the serial number is 7 as well.
I also ran the ldapsearch command against the other two servers and
they also showed a serial number of 7.

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411

.com>

Post by Michael Plemmons

Post by Michael Plemmons
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

Post by Michael Plemmons
On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and

ipa12 do not

Post by Michael Plemmons
have agreements between each other.
It appears that either ipa12.mgmt lost some level of its

replication

Post by Michael Plemmons
agreement with ipa13. I saw some level because users /

hosts were

Post by Michael Plemmons
replicated between all systems but we started seeing DNS

was not

Post by Michael Plemmons
resolving properly from ipa12. I do not know when this

started.

Post by Michael Plemmons
When looking at replication agreements on ipa12 I did not

see any

Post by Michael Plemmons
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show

has master.

Post by Michael Plemmons
When I run ipa-replica-manage ipa11.mgmt I see ipa13.mgmt

is a replica.

Post by Michael Plemmons
When I run ipa-replica-manage ipa12.mgmt nothing

returned.

Post by Michael Plemmons

Post by Michael Plemmons
I ran ipa-replica-manage connect --cacert=/etc/ipa/ca.crt
ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>

<http://ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.co

m>>

Post by Michael Plemmons

Post by Michael Plemmons
ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>> on ipa12.mgmt

Post by Michael Plemmons
I then ran the following
ipa-replica-manage force-sync --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
ipa-replica-manage re-initialize --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
I was still seeing bad DNS returns when dig'ing against

ipa12.mgmt.

Post by Michael Plemmons
I was able to create user and DNS records and see the

information

Post by Michael Plemmons
replicated properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl

start on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt because I wanted to make sure everything was

running

Post by Michael Plemmons
fresh after the changes above. While IPA was staring up

(DNS

Post by Michael Plemmons

Post by Michael Plemmons
started) we were able to see valid DNS queries returned

but

Post by Michael Plemmons

Post by Michael Plemmons
pki-tomcat would not start.
I am not sure what I need to do in order to get this

working. I

Post by Michael Plemmons
have included the output of certutil and getcert below

from all

Post by Michael Plemmons
three servers as well as the debug output for pki.
While the IPA system is coming up I am able to

successfully run

Post by Michael Plemmons
ldapsearch -x as the root user and see results. I am

also

Post by Michael Plemmons
able to

Post by Michael Plemmons
login with the "cn=Directory Manager" account and see

results.

Post by Michael Plemmons

Post by Michael Plemmons
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: =====

DEBUG

Post by Michael Plemmons

Post by Michael Plemmons
SUBSYSTEM INITIALIZED =======
============================================

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=debug
initialized debug
initSubsystem id=log

ready to

Post by Michael Plemmons
init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating

RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/c

a_audit)

Post by Michael Plemmons

Post by Michael Plemmons
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/p

ki-tomcat/logs/ca/transactions)

Post by Michael Plemmons
restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=log
initialized log
initSubsystem id=jss

ready to

Post by Michael Plemmons
init id=jss

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=jss
initialized jss
initSubsystem id=dbs

ready to

Post by Michael Plemmons
init id=dbs

DBSubsystem: init()

Post by Michael Plemmons
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
init
LdapBoundConnFactory:doCloning true

LdapAuthInfo: init()
LdapAuthInfo: init begins
LdapAuthInfo: init ends

before

Post by Michael Plemmons

Post by Michael Plemmons
makeConnection errorIfDown is true
errorIfDown true
SSLClientCertificateSelectionCB: Setting desired cert
subsystemCert cert-pki-ca

LdapJssSSLSocket: set

Post by Michael Plemmons
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL

handshake happened

Post by Michael Plemmons
Could not connect to LDAP server host

ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> port 636 Error

Post by Michael Plemmons
netscape.ldap.LDAPException: Authentication failed (48)
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConne

ction(LdapBoundConnFactory.java:205)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap

BoundConnFactory.java:166)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap

BoundConnFactory.java:130)

Post by Michael Plemmons

Post by Michael Plemmons
at

654)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.

java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine

.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS

ervlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce

ssorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe

thodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(Subject

.java:549)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(SecurityUt

il.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(Standar

dWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(Standar

dWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(StandardWrappe

r.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(Stand

ardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(Stand

ardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(LifecycleBase.j

ava:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(Cont

ainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(ContainerB

ase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru

n(ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru

n(ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(ContainerBas

e.java:873)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(StandardHost.

java:652)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(Host

Config.java:679)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.run(

HostConfig.java:1966)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.call(Executor

s.java:511)

Post by Michael Plemmons

Post by Michael Plemmons
at java.util.concurrent.FutureTas

k.run(FutureTask.java:266)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool

Executor.java:1142)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo

lExecutor.java:617)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect

to LDAP

Post by Michael Plemmons

Post by Michael Plemmons
server host ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com> <

http://ipa12.mgmt.crosschx.com

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com>>

Authentication

Post by Michael Plemmons
failed (48)

Post by Michael Plemmons
at

676)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.

java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine

.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS

ervlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce

ssorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe

thodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(Subject

.java:549)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(SecurityUt

il.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(Standar

dWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(Standar

dWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(StandardWrappe

r.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(Stand

ardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(Stand

ardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(LifecycleBase.j

ava:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(Cont

ainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(ContainerB

ase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru

n(ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru

n(ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(ContainerBas

e.java:873)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(StandardHost.

java:652)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(Host

Config.java:679)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.run(

HostConfig.java:1966)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.call(Executor

s.java:511)

Post by Michael Plemmons

Post by Michael Plemmons
at java.util.concurrent.FutureTas

k.run(FutureTask.java:266)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool

Executor.java:1142)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo

lExecutor.java:617)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)

CMSEngine.shutdown()

Post by Michael Plemmons
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCH

X-COM/

Post by Michael Plemmons

Post by Michael Plemmons
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI

Server-Cert

Post by Michael Plemmons
u,u,u MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C

Post by Michael Plemmons
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/

Certificate

Post by Michael Plemmons
Nickname Trust Attributes SSL,S/MIME,JAR/XPI

caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca CTu,Cu,Cu auditSigningCert cert-pki-ca u,u,Pu
ocspSigningCert cert-pki-ca u,u,u subsystemCert

cert-pki-ca u,u,u

Post by Michael Plemmons
Server-Cert cert-pki-ca u,u,u IPA13.MGMT (root)>certutil

-L -d

Post by Michael Plemmons

Post by Michael Plemmons
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate

Nickname

Post by Michael Plemmons
Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C

(root)>certutil -L -d

Post by Michael Plemmons

Post by Michael Plemmons
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
IPA12.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate

Nickname

Post by Michael Plemmons
Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA C,, (root)>certutil

-L -d

Post by Michael Plemmons

Post by Michael Plemmons
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
=================================================

IPA11.MGMT

Post by Michael Plemmons

Post by Michael Plemmons
(root)>getcert list Number of certificates and requests

being

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slap

d-MGMT-CROSSCHX-COM/pwdfile.txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <

http://ipa11.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:43

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229155652': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <

http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiatio

n,keyCertSign,cRLSign

Post by Michael Plemmons
pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <

http://ipa11.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:56:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientA

uth,id-kp-emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa11.mgmt.crosschx.com <

http://ipa11.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:46

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd

track: yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
================================== IPA13.MGMT

(root)>getcert list

Post by Michael Plemmons
Number of certificates and requests being tracked: 8.

Request ID

Post by Michael Plemmons
'20161229143449': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slap

d-MGMT-CROSSCHX-COM/pwdfile.txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <

http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229143826': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <

http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiatio

n,keyCertSign,cRLSign

Post by Michael Plemmons
pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <

http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 14:37:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientA

uth,id-kp-emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa13.mgmt.crosschx.com <

http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:23

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd

track: yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
=========================== IPA12.MGMT (root)>getcert

list

Post by Michael Plemmons
Number of

Post by Michael Plemmons
certificates and requests being tracked: 8. Request ID
'20161229151518': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slap

d-MGMT-CROSSCHX-COM/pwdfile.txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <

http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:51

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229151850': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Audit,O=MGMT.CROSSCHX.COM <

http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiatio

n,keyCertSign,cRLSign

Post by Michael Plemmons
pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <

http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:18:16

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientA

uth,id-kp-emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=ipa12.mgmt.crosschx.com <

http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd

track: yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

[Michael Plemmons]

I have done more searching in my logs and I see the following errors.

This is in the localhost log file /var/lib/pki/pki-tomcat/logs

May 15, 2017 3:08:08 PM org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
java.lang.NullPointerException

May 15, 2017 3:08:08 PM org.apache.catalina.core.StandardContext
loadOnStartup
SEVERE: Servlet [castart] in web application [/ca] threw load() exception
java.lang.NullPointerException

May 15, 2017 3:08:09 PM org.apache.catalina.core.StandardHostValve invoke
SEVERE: Exception Processing /ca/admin/ca/getStatus
javax.ws.rs.ServiceUnavailableException: Subsystem unavailable


Looking at the debug log it says Authentication failed for port 636.

[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init()
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init begins
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init ends
[15/May/2017:17:39:25][localhost-startStop-1]: init: before makeConnection
errorIfDown is true
[15/May/2017:17:39:25][localhost-startStop-1]: makeConnection: errorIfDown
true
[15/May/2017:17:39:25][localhost-startStop-1]:
SSLClientCertificateSelectionCB: Setting desired cert nickname to:
subsystemCert cert-pki-ca
[15/May/2017:17:39:25][localhost-startStop-1]: LdapJssSSLSocket: set client
auth cert nickname subsystemCert cert-pki-ca
[15/May/2017:17:39:25][localhost-startStop-1]:
SSLClientCertificatSelectionCB: Entering!
[15/May/2017:17:39:25][localhost-startStop-1]:
SSLClientCertificateSelectionCB: returning: null
[15/May/2017:17:39:25][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host ipa12.mgmt.crosschx.com port 636
Error netscape.ldap.LDAPException: Authentication failed (48)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)


I looked at the validity of the cert it mentions and it is fine.

(root)>getcert status -v -d /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca'
State MONITORING, stuck: no.


I then looked at the ldap errors around the time of this failure and I am
seeing this log entry.


[15/May/2017:17:38:42.063080758 +0000] set_krb5_creds - Could not get
initial credentials for principal [ldap/
***@MGMT.CROSSCHX.COM] in keytab
[FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC for
requested realm)

When I perform a klist against that keytab nothing appears out of the
ordinary compared to working IPA servers.

I am not sure what to look at next.





*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

On Wed, May 10, 2017 at 3:35 PM, Michael Plemmons <

Post by Michael Plemmons
The PKI service came up successfully but only when it uses BasicAuth
rather than SSL auth. I am not sure about what I need to do in order to
get the auth working over SSL again.
None of the certs are expired when I run getcert list and ipa-getcert list.
Since the failure is with attempts to login to LDAP over 636. I have been
attempting to auth to LDAP via port 636 and the ldapsearch is not
completing. When looking at packet captures, I see some the TCP handshake
and what appears to be the start of a SSL process and then everything hangs.
What is the proper method to test performing a ldapsearch over 636? Also,
the CS.cfg shows it wants to auth as cn=Directory Manager. I can
successfully auth with cn=Directory Manager over 389 but I think I am not
performing ldapsearch over 636 correctly.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com
On Fri, May 5, 2017 at 3:33 PM, Michael Plemmons <

Post by Michael Plemmons
I think I found the email thread. Asking for help with crashed freeIPA
istance. That email pointed to this link, https://www.redhat.com/a
rchives/freeipa-users/2017-January/msg00215.html. That link talked
about changing the CS.cfg file to use port 389 for PKI to auth to LDAP. I
made the necessary changes and PKI came up successfully.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com
On Fri, May 5, 2017 at 3:19 PM, Michael Plemmons <

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
www.crosschx.com

Post by Rob Crittenden

Post by Michael Plemmons
I just realized that I sent the reply directly to Rob and not to the
list. My response is inline

Ok, this is actually good news.
I made a similar proposal in another case and I was completely wrong.
Flo had the user do something and it totally fixed their auth error, I
just can't remember what it was or find the e-mail thread. I'm pretty
sure it was this calendar year though.
rob

Do you or Flo know what I could search for in the past emails to find
the answer to the problem?

Post by Rob Crittenden

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Thu, May 4, 2017 at 9:24 AM, Rob Crittenden <

Post by Michael Plemmons
I realized that I was not very clear in my statement about

testing with

Post by Michael Plemmons
ldapsearch. I had initially run it without logging in with

a

Post by Michael Plemmons
DN. I was

Post by Michael Plemmons
just running the local ldapsearch -x command. I then

tested on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt and ipa11.mgmt logging in with a full DN for the

admin and

Post by Michael Plemmons
"cn=Directory Manager" from ipa12.mgmt (broken server) and

ipa11.mgmt

Post by Michael Plemmons
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt and ipa11.mgmt as a non

root user.

Post by Michael Plemmons
I also ran the command showing a line count for the output

and

Post by Michael Plemmons
the line

Post by Michael Plemmons
counts for each were the same when run from ipa12.mgmt and

ipa11.mgmt.

Post by Michael Plemmons
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "DN" -w PASSWORD -b

Post by Michael Plemmons
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> -D "cn=directory manager"

-w

Post by Michael Plemmons
PASSWORD dn
The CA has its own suffix and replication agreements. Given

the auth

Post by Michael Plemmons
error and recent (5 months) renewal of CA credentials I'd

check

Post by Michael Plemmons
that the
CA agent authentication entries are correct.
$ ldapsearch -LLL -x -D 'cn=directory manager' -W -b
uid=ipara,ou=people,o=ipaca description
The format is 2;serial#,subject,issuer
# certutil -L -d /etc/httpd/alias -n ipaCert |grep Serial
The serial # should match that in the description everywhere.
rob
On the CA (IPA13.MGMT) I ran the ldapsearch command and see that

the

Post by Michael Plemmons
serial number is 7. I then ran the certutil command on all three
servers and the serial number is 7 as well.
I also ran the ldapsearch command against the other two servers

and

Post by Michael Plemmons
they also showed a serial number of 7.

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411

.com>

Post by Michael Plemmons

Post by Michael Plemmons
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

Post by Michael Plemmons
On Wed, May 3, 2017 at 5:28 PM, Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months ago
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and ipa12. ipa11 and

ipa12 do not

Post by Michael Plemmons
have agreements between each other.
It appears that either ipa12.mgmt lost some level of its

replication

Post by Michael Plemmons
agreement with ipa13. I saw some level because users /

hosts were

Post by Michael Plemmons
replicated between all systems but we started seeing DNS

was not

Post by Michael Plemmons
resolving properly from ipa12. I do not know when this

started.

Post by Michael Plemmons
When looking at replication agreements on ipa12 I did

not

Post by Michael Plemmons
see any

Post by Michael Plemmons
agreement with ipa13.
When I run ipa-replica-manage list all three hosts show

has master.

Post by Michael Plemmons
When I run ipa-replica-manage ipa11.mgmt I see

ipa13.mgmt

Post by Michael Plemmons
is a replica.

Post by Michael Plemmons
When I run ipa-replica-manage ipa12.mgmt nothing

returned.

Post by Michael Plemmons

Post by Michael Plemmons
I ran ipa-replica-manage connect

--cacert=/etc/ipa/ca.crt

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com <

http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons

Post by Michael Plemmons
ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>> on ipa12.mgmt

Post by Michael Plemmons
I then ran the following
ipa-replica-manage force-sync --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
ipa-replica-manage re-initialize --from

ipa13.mgmt.crosschx.com <http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
I was still seeing bad DNS returns when dig'ing against

ipa12.mgmt.

Post by Michael Plemmons
I was able to create user and DNS records and see the

information

Post by Michael Plemmons
replicated properly across all three nodes.
I then ran ipactl stop on ipa12.mgmt and then ipactl

start on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt because I wanted to make sure everything was

running

Post by Michael Plemmons
fresh after the changes above. While IPA was staring

up (DNS

Post by Michael Plemmons

Post by Michael Plemmons
started) we were able to see valid DNS queries returned

but

Post by Michael Plemmons

Post by Michael Plemmons
pki-tomcat would not start.
I am not sure what I need to do in order to get this

working. I

Post by Michael Plemmons
have included the output of certutil and getcert below

from all

Post by Michael Plemmons
three servers as well as the debug output for pki.
While the IPA system is coming up I am able to

successfully run

Post by Michael Plemmons
ldapsearch -x as the root user and see results. I am

also

Post by Michael Plemmons
able to

Post by Michael Plemmons
login with the "cn=Directory Manager" account and see

results.

Post by Michael Plemmons

Post by Michael Plemmons
The debug log shows the following error.
============================================
[03/May/2017:21:22:01][localhost-startStop-1]: =====

DEBUG

Post by Michael Plemmons

Post by Michael Plemmons
SUBSYSTEM INITIALIZED =======
============================================

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=debug
initialized debug
initSubsystem id=log

ready to

Post by Michael Plemmons
init id=log
[03/May/2017:21:22:01][localhost-startStop-1]: Creating

RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/c

a_audit)

Post by Michael Plemmons

Post by Michael Plemmons
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/p

ki-tomcat/logs/ca/transactions)

Post by Michael Plemmons
restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=log
initialized log
initSubsystem id=jss

ready to

Post by Michael Plemmons
init id=jss

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?
/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

about to

Post by Michael Plemmons
look for cert for auto-shutdown support:auditSigningCert

cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=jss
initialized jss
initSubsystem id=dbs

ready to

Post by Michael Plemmons
init id=dbs

DBSubsystem: init()

Post by Michael Plemmons
mEnableSerialMgmt=true
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
LdapBoundConnFactor(DBSubsystem)
init
LdapBoundConnFactory:doCloning true

LdapAuthInfo: init()
LdapAuthInfo: init begins
LdapAuthInfo: init ends

before

Post by Michael Plemmons

Post by Michael Plemmons
makeConnection errorIfDown is true
errorIfDown true
SSLClientCertificateSelectionCB: Setting desired cert
subsystemCert cert-pki-ca

LdapJssSSLSocket: set

Post by Michael Plemmons
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL

handshake happened

Post by Michael Plemmons
Could not connect to LDAP server host

ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>> port 636 Error

Post by Michael Plemmons
netscape.ldap.LDAPException: Authentication failed (48)
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConne

ction(LdapBoundConnFactory.java:205)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap

BoundConnFactory.java:166)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(Ldap

BoundConnFactory.java:130)

Post by Michael Plemmons

Post by Michael Plemmons
at

654)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.

java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine

.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS

ervlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccess

orImpl.invoke0(Native

Post by Michael Plemmons
Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce

ssorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe

thodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(Subject

.java:549)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(SecurityUt

il.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(Standar

dWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(Standar

dWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(StandardWrappe

r.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(Stand

ardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(Stand

ardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(LifecycleBase.j

ava:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(Cont

ainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(ContainerB

ase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru

n(ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru

n(ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(ContainerBas

e.java:873)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(StandardHost.

java:652)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(Host

Config.java:679)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.run(

HostConfig.java:1966)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.call(Executor

s.java:511)

Post by Michael Plemmons

Post by Michael Plemmons
at java.util.concurrent.FutureTas

k.run(FutureTask.java:266)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool

Executor.java:1142)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo

lExecutor.java:617)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)
Internal Database Error encountered: Could not connect

to LDAP

Post by Michael Plemmons

Post by Michael Plemmons
server host ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com> <

http://ipa12.mgmt.crosschx.com

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com>>

Authentication

Post by Michael Plemmons
failed (48)

Post by Michael Plemmons
at

676)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.

java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine

.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons
at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
at

com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartS

ervlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons
at sun.reflect.NativeMethodAccess

orImpl.invoke0(Native

Post by Michael Plemmons
Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce

ssorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe

thodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.reflect.Method.invoke(Method.java:498)
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil

.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>AsPrivileged(Subject

.java:549)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(SecurityUt

il.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(Secu

rityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(Standar

dWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(Standar

dWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(StandardWrappe

r.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(Stand

ardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(Stand

ardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(LifecycleBase.j

ava:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(Cont

ainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(ContainerB

ase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru

n(ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.ru

n(ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at java.security.AccessController.doPrivileged(Native

Method)

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChild(ContainerBas

e.java:873)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardHost.addChild(StandardHost.

java:652)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig.deployDescriptor(Host

Config.java:679)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.startup.HostConfig$DeployDescriptor.run(

HostConfig.java:1966)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.Executors$RunnableAdapter.call(Executor

s.java:511)

Post by Michael Plemmons

Post by Michael Plemmons
at java.util.concurrent.FutureTas

k.run(FutureTask.java:266)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool

Executor.java:1142)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo

lExecutor.java:617)

Post by Michael Plemmons

Post by Michael Plemmons
at java.lang.Thread.run(Thread.java:745)

CMSEngine.shutdown()

Post by Michael Plemmons
=============================
IPA11.MGMT
(root)>certutil -L -d /etc/dirsrv/slapd-MGMT-CROSSCH

X-COM/

Post by Michael Plemmons

Post by Michael Plemmons
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI

Server-Cert

Post by Michael Plemmons
u,u,u MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C

Post by Michael Plemmons
(root)>certutil -L -d /var/lib/pki/pki-tomcat/alias/

Certificate

Post by Michael Plemmons
Nickname Trust Attributes SSL,S/MIME,JAR/XPI

caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca CTu,Cu,Cu auditSigningCert cert-pki-ca

u,u,Pu

Post by Michael Plemmons

Post by Michael Plemmons
ocspSigningCert cert-pki-ca u,u,u subsystemCert

cert-pki-ca u,u,u

Post by Michael Plemmons
Server-Cert cert-pki-ca u,u,u IPA13.MGMT

(root)>certutil -L -d

Post by Michael Plemmons

Post by Michael Plemmons
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate

Nickname

Post by Michael Plemmons
Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA CT,C,C

(root)>certutil -L -d

Post by Michael Plemmons

Post by Michael Plemmons
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
IPA12.MGMT (root)>certutil -L -d
/etc/dirsrv/slapd-MGMT-CROSSCHX-COM/ Certificate

Nickname

Post by Michael Plemmons
Trust

Post by Michael Plemmons
Attributes SSL,S/MIME,JAR/XPI Server-Cert u,u,u

MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> IPA CA C,, (root)>certutil

-L -d

Post by Michael Plemmons

Post by Michael Plemmons
/var/lib/pki/pki-tomcat/alias/ Certificate Nickname

Trust

Post by Michael Plemmons
Attributes

Post by Michael Plemmons
SSL,S/MIME,JAR/XPI caSigningCert cert-pki-ca CTu,Cu,Cu
auditSigningCert cert-pki-ca u,u,Pu ocspSigningCert

cert-pki-ca

Post by Michael Plemmons
u,u,u subsystemCert cert-pki-ca u,u,u Server-Cert

cert-pki-ca u,u,u

Post by Michael Plemmons
=================================================

IPA11.MGMT

Post by Michael Plemmons

Post by Michael Plemmons
(root)>getcert list Number of certificates and requests

being

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slap

d-MGMT-CROSSCHX-COM/pwdfile.txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=ipa11.mgmt.crosschx.com <

http://ipa11.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:43

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229155652': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=CA Audit,O=MGMT.CROSSCHX.COM <

http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
digitalSignature,nonRepudiation,keyCertSign,cRLSign
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"subsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiatio

n,keyCertSign,cRLSign

Post by Michael Plemmons
pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert

"caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=ipa11.mgmt.crosschx.com <

http://ipa11.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:56:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientA

uth,id-kp-emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=ipa11.mgmt.crosschx.com <

http://ipa11.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa11.mgmt.crosschx.com

<http://ipa11.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:52:46

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd

track: yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM

digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
================================== IPA13.MGMT

(root)>getcert list

Post by Michael Plemmons
Number of certificates and requests being tracked: 8.

Request ID

Post by Michael Plemmons
'20161229143449': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slap

d-MGMT-CROSSCHX-COM/pwdfile.txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=ipa13.mgmt.crosschx.com <

http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:20

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229143826': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=CA Audit,O=MGMT.CROSSCHX.COM <

http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
digitalSignature,nonRepudiation,keyCertSign,cRLSign
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"subsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiatio

n,keyCertSign,cRLSign

Post by Michael Plemmons
pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert

"caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=ipa13.mgmt.crosschx.com <

http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 14:37:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientA

uth,id-kp-emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=ipa13.mgmt.crosschx.com <

http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 14:34:23

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd

track: yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM

digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
=========================== IPA12.MGMT (root)>getcert

list

Post by Michael Plemmons
Number of

Post by Michael Plemmons
certificates and requests being tracked: 8. Request ID
'20161229151518': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate
DB',pinfile='/etc/dirsrv/slap

d-MGMT-CROSSCHX-COM/pwdfile.txt'

Post by Michael Plemmons
type=NSSDB,location='/etc/dirsrv/slapd-MGMT-CROSSCHX-COM',ni

ckname='Server-Cert',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=ipa12.mgmt.crosschx.com <

http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:51

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_dirsrv
MGMT-CROSSCHX-COM track: yes auto-renew: yes Request ID
'20161229151850': status: MONITORING stuck: no key pair

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='au

ditSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=CA Audit,O=MGMT.CROSSCHX.COM <

http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
digitalSignature,nonRepudiation

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"auditSigningCert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='oc

spSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=OCSP Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
digitalSignature,nonRepudiation,keyCertSign,cRLSign
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"ocspSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='su

bsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=CA Subsystem,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM> <http://MGMT.CROSSCHX.COM>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save
/usr/libexec/ipa/certmonger/stop_pkicad post-save
/usr/libexec/ipa/certmonger/renew_ca_cert

"subsystemCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca',token='NSS Certificate DB',pin set

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ca

SigningCert

Post by Michael Plemmons

Post by Michael Plemmons
dogtag-ipa-ca-renew-agent issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=Certificate Authority,O=MGMT.CROSSCHX.COM

<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2036-11-22 13:00:25

UTC key

Post by Michael Plemmons
usage: digitalSignature,nonRepudiatio

n,keyCertSign,cRLSign

Post by Michael Plemmons
pre-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/stop_pkicad

post-save

Post by Michael Plemmons

Post by Michael Plemmons
/usr/libexec/ipa/certmonger/renew_ca_cert

"caSigningCert

Post by Michael Plemmons

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Se

rver-Cert

Post by Michael Plemmons
cert-pki-ca',token='NSS
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=ipa12.mgmt.crosschx.com <

http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-19 15:18:16

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientA

uth,id-kp-emailProtection

Post by Michael Plemmons

Post by Michael Plemmons
pre-save command: /usr/libexec/ipa/certmonger/st

op_pkicad

Post by Michael Plemmons
post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/renew_ca_cert

"Server-Cert

Post by Michael Plemmons
cert-pki-ca" track: yes auto-renew: yes Request ID

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert

',token='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB' CA: IPA issuer: CN=Certificate
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=ipa12.mgmt.crosschx.com <

http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>,O=MGMT.CROSSCHX.COM
<http://MGMT.CROSSCHX.COM>

Post by Michael Plemmons
<http://MGMT.CROSSCHX.COM> expires: 2018-12-30 15:14:54

UTC key
digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save

post-save

Post by Michael Plemmons
command: /usr/libexec/ipa/certmonger/restart_httpd

track: yes

Post by Michael Plemmons
MONITORING
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons

Post by Michael Plemmons
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'

type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',to

ken='NSS

Post by Michael Plemmons
CN=Certificate

Post by Michael Plemmons
Authority,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM
CN=IPA RA,O=MGMT.CROSSCHX.COM <http://MGMT.CROSSCHX.COM

digitalSignature,nonRepudiation,keyEncipherment,dataEncipher

ment

Post by Michael Plemmons

Post by Michael Plemmons
eku: id-kp-serverAuth,id-kp-clientAuth pre-save
/usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save
/usr/libexec/ipa/certmonger/renew_ra_cert track: yes

auto-renew: yes

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

[Florence Blanc-Renaud]

This post might be inappropriate. Click to display it.

[Michael Plemmons]

*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

Post by Michael Plemmons
I have done more searching in my logs and I see the following errors.
This is in the localhost log file /var/lib/pki/pki-tomcat/logs
May 15, 2017 3:08:08 PM org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
java.lang.NullPointerException
May 15, 2017 3:08:08 PM org.apache.catalina.core.StandardContext
loadOnStartup
SEVERE: Servlet [castart] in web application [/ca] threw load() exception
java.lang.NullPointerException
May 15, 2017 3:08:09 PM org.apache.catalina.core.StandardHostValve invoke
SEVERE: Exception Processing /ca/admin/ca/getStatus
javax.ws.rs <http://javax.ws.rs>.ServiceUnavailableException: Subsystem
unavailable
Looking at the debug log it says Authentication failed for port 636.
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init()
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init begins
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init ends
[15/May/2017:17:39:25][localhost-startStop-1]: init: before
makeConnection errorIfDown is true
errorIfDown true
subsystemCert cert-pki-ca
[15/May/2017:17:39:25][localhost-startStop-1]: LdapJssSSLSocket: set
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[15/May/2017:17:39:25][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com> port 636 Error
netscape.ldap.LDAPException: Authentication failed (48)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConne
ction(LdapBoundConnFactory.java:205)
I looked at the validity of the cert it mentions and it is fine.
(root)>getcert status -v -d /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca'
State MONITORING, stuck: no.
I then looked at the ldap errors around the time of this failure and I
am seeing this log entry.
[15/May/2017:17:38:42.063080758 +0000] set_krb5_creds - Could not get
initial credentials for principal
[FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC for
requested realm)
When I perform a klist against that keytab nothing appears out of the
ordinary compared to working IPA servers.
I am not sure what to look at next.

Hi,
you can try the following to manually replay the connection established by
root$ export LDAPTLS_CACERTDIR=/etc/pki/pki-tomcat/alias
root$ export LDAPTLS_CERT='subsystemCert cert-pki-ca'
The above commands specify the NSSDB containing the user certificate and
its name for SASL-EXTERNAL authentication.
Then note the value obtained below as it will be used for the next step as
root$ grep internal /etc/pki/pki-tomcat/password.conf
internal=<some value>
root$ ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL -Q
-LLL dn namingcontexts
<<<< here supply the value found above
namingcontexts: cn=changelog
namingcontexts: dc=ipadomain,dc=com
namingcontexts: o=ipaca

So I guess I found my problem.

(root)>ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL -Q
-LLL dn namingcontexts
Please enter pin, password, or pass phrase for security token 'ldap(0)':
ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
additional info: TLS error -12195:Peer does not recognize and trust the
CA that issued your certificate.


I looked at our certs in /etc/dirsrv/slapd-IPADOMAIN-COM and found the
following.

IPA12 - problem server
(root)>certutil -L -d /etc/dirsrv/slapd-IPADOMAIN-COM

Certificate Nickname Trust
Attributes

SSL,S/MIME,JAR/XPI

Server-Cert u,u,u
IPADOMAIN-COM IPA CA C,,



IPA11/IPA13 - 11 was the master and 13 is the new master
(root)>certutil -L -d /etc/dirsrv/slapd-IPADOMAIN-COM

Certificate Nickname Trust
Attributes

SSL,S/MIME,JAR/XPI

Server-Cert u,u,u
IPADOMAIN-COM IPA CA CT,C,C

In the LDAP server access log (in /etc/dirsrv/slapd-IPADOMAIN.COM/access),
[18/May/2017:13:35:14.822090417 +0200] conn=297 fd=150 slot=150 SSL
connection from xxx to yyy
[18/May/2017:13:35:15.789414017 +0200] conn=297 TLS1.2 128-bit AES-GCM;
client CN=CA Subsystem,O=IPADOMAIN.COM; issuer CN=Certificate Authority,O=
IPADOMAIN.COM
[18/May/2017:13:35:15.793108509 +0200] conn=297 TLS1.2 client bound as
uid=pkidbuser,ou=people,o=ipaca
[18/May/2017:13:35:15.798101505 +0200] conn=297 op=0 BIND dn=""
method=sasl version=3 mech=EXTERNAL
[18/May/2017:13:35:15.800322076 +0200] conn=297 op=0 RESULT err=0 tag=97
nentries=0 etime=0 dn="uid=pkidbuser,ou=people,o=ipaca"
HTH,
Flo.

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Wed, May 10, 2017 at 3:35 PM, Michael Plemmons
The PKI service came up successfully but only when it uses BasicAuth
rather than SSL auth. I am not sure about what I need to do in
order to get the auth working over SSL again.
None of the certs are expired when I run getcert list and
ipa-getcert list.
Since the failure is with attempts to login to LDAP over 636. I
have been attempting to auth to LDAP via port 636 and the ldapsearch
is not completing. When looking at packet captures, I see some the
TCP handshake and what appears to be the start of a SSL process and
then everything hangs.
What is the proper method to test performing a ldapsearch over 636?
Also, the CS.cfg shows it wants to auth as cn=Directory Manager. I
can successfully auth with cn=Directory Manager over 389 but I think
I am not performing ldapsearch over 636 correctly.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Fri, May 5, 2017 at 3:33 PM, Michael Plemmons
I think I found the email thread. Asking for help with crashed
freeIPA istance. That email pointed to this
link, https://www.redhat.com/archives/freeipa-users/2017-January/
msg00215.html
<https://www.redhat.com/archives/freeipa-users/2017-January/
msg00215.html>.
That link talked about changing the CS.cfg file to use port 389
for PKI to auth to LDAP. I made the necessary changes and PKI
came up successfully.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Fri, May 5, 2017 at 3:19 PM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
On Fri, May 5, 2017 at 3:15 PM, Rob Crittenden

Post by Michael Plemmons
I just realized that I sent the reply directly to Rob

and not to the

Post by Michael Plemmons
list. My response is inline

Ok, this is actually good news.
I made a similar proposal in another case and I was
completely wrong.
Flo had the user do something and it totally fixed their
auth error, I
just can't remember what it was or find the e-mail
thread. I'm pretty
sure it was this calendar year though.
rob
Do you or Flo know what I could search for in the past
emails to find the answer to the problem?

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

Post by Michael Plemmons
On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com/>

Post by Michael Plemmons
On Thu, May 4, 2017 at 9:24 AM, Rob Crittenden

Post by Michael Plemmons
I realized that I was not very clear in my

statement about

Post by Michael Plemmons
testing with

Post by Michael Plemmons
ldapsearch. I had initially run it without

logging in with a

Post by Michael Plemmons
DN. I was

Post by Michael Plemmons
just running the local ldapsearch -x

command. I then tested on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt and ipa11.mgmt logging in with a

full DN for the

Post by Michael Plemmons
admin and

Post by Michael Plemmons
"cn=Directory Manager" from ipa12.mgmt

(broken server) and

Post by Michael Plemmons
ipa11.mgmt

Post by Michael Plemmons
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt and

ipa11.mgmt as a non

Post by Michael Plemmons
root user.

Post by Michael Plemmons
I also ran the command showing a line count

for the output and

Post by Michael Plemmons
the line

Post by Michael Plemmons
counts for each were the same when run from

ipa12.mgmt and

Post by Michael Plemmons
ipa11.mgmt.

Post by Michael Plemmons
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>> -D "DN" -w PASSWORD -b
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn

Post by Michael Plemmons

Post by Michael Plemmons
ldapsearch -LLL -h ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>> -D "cn=directory
manager" -w

Post by Michael Plemmons
PASSWORD dn
The CA has its own suffix and replication

agreements. Given the auth

Post by Michael Plemmons
error and recent (5 months) renewal of CA

credentials I'd check

Post by Michael Plemmons
that the
CA agent authentication entries are correct.
$ ldapsearch -LLL -x -D 'cn=directory manager'

-W -b

Post by Michael Plemmons
uid=ipara,ou=people,o=ipaca description
The format is 2;serial#,subject,issuer
# certutil -L -d /etc/httpd/alias -n ipaCert

|grep Serial

Post by Michael Plemmons
The serial # should match that in the

description everywhere.

Post by Michael Plemmons
rob
On the CA (IPA13.MGMT) I ran the ldapsearch

command and see that the

Post by Michael Plemmons
serial number is 7. I then ran the certutil

command on all three

Post by Michael Plemmons
servers and the serial number is 7 as well.
I also ran the ldapsearch command against the

other two servers and

Post by Michael Plemmons
they also showed a serial number of 7.

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer |

CROSSCHX

Post by Michael Plemmons

Post by Michael Plemmons
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com>

Post by Michael Plemmons
<http://www.crosschx.com/>

Post by Michael Plemmons
On Wed, May 3, 2017 at 5:28 PM, Michael

Plemmons

Post by Michael Plemmons

Post by Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6 months

ago

Post by Michael Plemmons

Post by Michael Plemmons
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and

ipa12. ipa11 and

Post by Michael Plemmons
ipa12 do not

Post by Michael Plemmons
have agreements between each other.
It appears that either ipa12.mgmt lost

some level of its

Post by Michael Plemmons
replication

Post by Michael Plemmons
agreement with ipa13. I saw some level

because users /

Post by Michael Plemmons
hosts were

Post by Michael Plemmons
replicated between all systems but we

started seeing DNS

Post by Michael Plemmons
was not

Post by Michael Plemmons
resolving properly from ipa12. I do not

know when this

Post by Michael Plemmons
started.

Post by Michael Plemmons
When looking at replication agreements

on ipa12 I did not

Post by Michael Plemmons
see any

Post by Michael Plemmons
agreement with ipa13.
When I run ipa-replica-manage list all

three hosts show

Post by Michael Plemmons
has master.

Post by Michael Plemmons
When I run ipa-replica-manage ipa11.mgmt

I see ipa13.mgmt

Post by Michael Plemmons
is a replica.

Post by Michael Plemmons
When I run ipa-replica-manage ipa12.mgmt

nothing returned.

Post by Michael Plemmons

Post by Michael Plemmons
I ran ipa-replica-manage connect

--cacert=/etc/ipa/ca.crt

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>> on ipa12.mgmt

Post by Michael Plemmons

Post by Michael Plemmons
I then ran the following
ipa-replica-manage force-sync --from

ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
ipa-replica-manage re-initialize --from

ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
I was still seeing bad DNS returns when

dig'ing against

Post by Michael Plemmons
ipa12.mgmt.

Post by Michael Plemmons
I was able to create user and DNS

records and see the

Post by Michael Plemmons
information

Post by Michael Plemmons
replicated properly across all three

nodes.

Post by Michael Plemmons

Post by Michael Plemmons
I then ran ipactl stop on ipa12.mgmt and

then ipactl start on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt because I wanted to make sure

everything was

Post by Michael Plemmons
running

Post by Michael Plemmons
fresh after the changes above. While

IPA was staring up (DNS

Post by Michael Plemmons

Post by Michael Plemmons
started) we were able to see valid DNS

queries returned but

Post by Michael Plemmons

Post by Michael Plemmons
pki-tomcat would not start.
I am not sure what I need to do in order

to get this

Post by Michael Plemmons
working. I

Post by Michael Plemmons
have included the output of certutil and

getcert below

Post by Michael Plemmons
from all

Post by Michael Plemmons
three servers as well as the debug

output for pki.

Post by Michael Plemmons

Post by Michael Plemmons
While the IPA system is coming up I am

able to

Post by Michael Plemmons
successfully run

Post by Michael Plemmons
ldapsearch -x as the root user and see

results. I am also

Post by Michael Plemmons
able to

Post by Michael Plemmons
login with the "cn=Directory Manager"

account and see results.

Post by Michael Plemmons

Post by Michael Plemmons
The debug log shows the following error.
=============================

===============
[03/May/2017:21:22:01][localhost-startStop-1]: =====
DEBUG

Post by Michael Plemmons

Post by Michael Plemmons
SUBSYSTEM INITIALIZED =======
=============================

===============

Post by Michael Plemmons
restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?

/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

Post by Michael Plemmons
about to

Post by Michael Plemmons
look for cert for auto-shutdown

support:auditSigningCert

Post by Michael Plemmons
cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=debug
initialized debug
initSubsystem id=log

ready to

Post by Michael Plemmons
init id=log

[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pk
i-tomcat/logs/ca/signedAudit/ca_audit)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/p
ki-tomcat/logs/ca/transactions)

Post by Michael Plemmons
restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?

/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

Post by Michael Plemmons
about to

Post by Michael Plemmons
look for cert for auto-shutdown

support:auditSigningCert

Post by Michael Plemmons
cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=log
initialized log
initSubsystem id=jss

ready to

Post by Michael Plemmons
init id=jss

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?

/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

Post by Michael Plemmons
about to

Post by Michael Plemmons
look for cert for auto-shutdown

support:auditSigningCert

Post by Michael Plemmons
cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=jss
initialized jss
initSubsystem id=dbs

ready to

Post by Michael Plemmons
init id=dbs

DBSubsystem: init()

Post by Michael Plemmons
mEnableSerialMgmt=true

[03/May/2017:21:22:01][localhost-startStop-1]: Creating

Post by Michael Plemmons

Post by Michael Plemmons
LdapBoundConnFactor(DBSubsystem)
init
LdapBoundConnFactory:doCloning true

LdapAuthInfo: init()
LdapAuthInfo: init begins
LdapAuthInfo: init ends

[03/May/2017:21:22:01][localhost-startStop-1]: init: before

Post by Michael Plemmons

Post by Michael Plemmons
makeConnection errorIfDown is true
errorIfDown true
SSLClientCertificateSelectionCB: Setting

desired cert

Post by Michael Plemmons

Post by Michael Plemmons
subsystemCert cert-pki-ca

LdapJssSSLSocket: set

Post by Michael Plemmons
client auth cert nickname subsystemCert

cert-pki-ca

Post by Michael Plemmons

Post by Michael Plemmons
SSLClientCertificatSelectionCB: Entering!

returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL

Post by Michael Plemmons
handshake happened

Post by Michael Plemmons
Could not connect to LDAP server host

ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>>> port 636 Error
Authentication failed (48)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.
LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.
LdapBoundConnFactory.init(LdapBoundConnFactory.java:166)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.
LdapBoundConnFactory.init(LdapBoundConnFactory.java:130)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.dbs.DBSu
bsystem.init(DBSubsystem.java:654)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSE
ngine.initSubsystem(CMSEngine.java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSE
ngine.initSubsystems(CMSEngine.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMS
Engine.init(CMSEngine.java:571)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.certsrv.apps.CMS.init(CMS.java:187)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.certsrv.apps.CMS.start(CMS.java:1616)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cms.servlet.base.
CMSStartServlet.init(CMSStartServlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.
init(GenericServlet.java:158)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Post by Michael Plemmons
Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccess
orImpl.invoke(NativeMethodAccessorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAc
cessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.lang.reflect.Method.invoke(Method.java:498)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.S
ecurityUtil$1.run(SecurityUtil.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.S
ecurityUtil$1.run(SecurityUtil.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.security.AccessController.doPrivileged(Native

Post by Michael Plemmons
Method)

Post by Michael Plemmons
at javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>

Post by Michael Plemmons
<http://javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do

Post by Michael Plemmons

Post by Michael Plemmons
AsPrivileged(Subject.java:549)
at

org.apache.catalina.security.S
ecurityUtil.execute(SecurityUtil.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.S
ecurityUtil.doAsPrivilege(SecurityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.S
ecurityUtil.doAsPrivilege(SecurityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.Stand
ardWrapper.initServlet(StandardWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.Stand
ardWrapper.loadServlet(StandardWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.Stand
ardWrapper.load(StandardWrapper.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.Stand
ardContext.loadOnStartup(StandardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.Stand
ardContext.startInternal(StandardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.Lifec
ycleBase.start(LifecycleBase.java:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.Conta
inerBase.addChildInternal(ContainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.Conta
inerBase.access$000(ContainerBase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.Conta
inerBase$PrivilegedAddChild.run(ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.Conta
inerBase$PrivilegedAddChild.run(ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.security.AccessController.doPrivileged(Native

[Florence Blanc-Renaud]

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
I have done more searching in my logs and I see the following errors.
This is in the localhost log file /var/lib/pki/pki-tomcat/logs
May 15, 2017 3:08:08 PM
org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
java.lang.NullPointerException
May 15, 2017 3:08:08 PM org.apache.catalina.core.StandardContext
loadOnStartup
SEVERE: Servlet [castart] in web application [/ca] threw load() exception
java.lang.NullPointerException
May 15, 2017 3:08:09 PM
org.apache.catalina.core.StandardHostValve invoke
SEVERE: Exception Processing /ca/admin/ca/getStatus
javax.ws.rs <http://javax.ws.rs>
<http://javax.ws.rs>.ServiceUnavailableException: Subsystem
unavailable
Looking at the debug log it says Authentication failed for port 636.
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init()
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init begins
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init ends
[15/May/2017:17:39:25][localhost-startStop-1]: init: before
makeConnection errorIfDown is true
errorIfDown true
subsystemCert cert-pki-ca
[15/May/2017:17:39:25][localhost-startStop-1]: LdapJssSSLSocket: set
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[15/May/2017:17:39:25][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>> port 636 Error
netscape.ldap.LDAPException: Authentication failed (48)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)
I looked at the validity of the cert it mentions and it is fine.
(root)>getcert status -v -d /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca'
State MONITORING, stuck: no.
I then looked at the ldap errors around the time of this failure and I
am seeing this log entry.
[15/May/2017:17:38:42.063080758 +0000] set_krb5_creds - Could not get
initial credentials for principal
[FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC for
requested realm)
When I perform a klist against that keytab nothing appears out of the
ordinary compared to working IPA servers.
I am not sure what to look at next.
Hi,
you can try the following to manually replay the connection
root$ export LDAPTLS_CACERTDIR=/etc/pki/pki-tomcat/alias
root$ export LDAPTLS_CERT='subsystemCert cert-pki-ca'
The above commands specify the NSSDB containing the user certificate
and its name for SASL-EXTERNAL authentication.
Then note the value obtained below as it will be used for the next
root$ grep internal /etc/pki/pki-tomcat/password.conf
internal=<some value>
root$ ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL
-Q -LLL dn namingcontexts
Please enter pin, password, or pass phrase for security token
'ldap(0)': <<<< here supply the value found above
namingcontexts: cn=changelog
namingcontexts: dc=ipadomain,dc=com
namingcontexts: o=ipaca
So I guess I found my problem.
(root)>ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL -Q
-LLL dn namingcontexts
ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
additional info: TLS error -12195:Peer does not recognize and trust
the CA that issued your certificate.
I looked at our certs in /etc/dirsrv/slapd-IPADOMAIN-COM and found the
following.
IPA12 - problem server
(root)>certutil -L -d /etc/dirsrv/slapd-IPADOMAIN-COM
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,u
IPADOMAIN-COM IPA CA C,,
IPA11/IPA13 - 11 was the master and 13 is the new master
(root)>certutil -L -d /etc/dirsrv/slapd-IPADOMAIN-COM
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,u
IPADOMAIN-COM IPA CA CT,C,C

Good news! In this case the fix is trivial:
root$ certutil -M -d /etc/dirsrv/slapd-IPADOMAIN-COM -n 'IPADOMAIN-COM
IPA CA' -t CT,C,C

Flo.

Post by Michael Plemmons
In the LDAP server access log (in
/etc/dirsrv/slapd-IPADOMAIN.COM/access), you should see the
[18/May/2017:13:35:14.822090417 +0200] conn=297 fd=150 slot=150 SSL
connection from xxx to yyy
[18/May/2017:13:35:15.789414017 +0200] conn=297 TLS1.2 128-bit
AES-GCM; client CN=CA Subsystem,O=IPADOMAIN.COM
<http://IPADOMAIN.COM>; issuer CN=Certificate
Authority,O=IPADOMAIN.COM <http://IPADOMAIN.COM>
[18/May/2017:13:35:15.793108509 +0200] conn=297 TLS1.2 client bound
as uid=pkidbuser,ou=people,o=ipaca
[18/May/2017:13:35:15.798101505 +0200] conn=297 op=0 BIND dn=""
method=sasl version=3 mech=EXTERNAL
[18/May/2017:13:35:15.800322076 +0200] conn=297 op=0 RESULT err=0
tag=97 nentries=0 etime=0 dn="uid=pkidbuser,ou=people,o=ipaca"
HTH,
Flo.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>
<http://www.crosschx.com/>
On Wed, May 10, 2017 at 3:35 PM, Michael Plemmons
The PKI service came up successfully but only when it uses BasicAuth
rather than SSL auth. I am not sure about what I need to do in
order to get the auth working over SSL again.
None of the certs are expired when I run getcert list and
ipa-getcert list.
Since the failure is with attempts to login to LDAP over 636. I
have been attempting to auth to LDAP via port 636 and the ldapsearch
is not completing. When looking at packet captures, I see some the
TCP handshake and what appears to be the start of a SSL process and
then everything hangs.
What is the proper method to test performing a ldapsearch over 636?
Also, the CS.cfg shows it wants to auth as cn=Directory Manager. I
can successfully auth with cn=Directory Manager over 389 but I think
I am not performing ldapsearch over 636 correctly.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>
<http://www.crosschx.com/>
On Fri, May 5, 2017 at 3:33 PM, Michael Plemmons
I think I found the email thread. Asking for help with crashed
freeIPA istance. That email pointed to this
link,
https://www.redhat.com/archives/freeipa-users/2017-January/msg00215.html
<https://www.redhat.com/archives/freeipa-users/2017-January/msg00215.html>
<https://www.redhat.com/archives/freeipa-users/2017-January/msg00215.html
<https://www.redhat.com/archives/freeipa-users/2017-January/msg00215.html>>.
That link talked about changing the CS.cfg file to use port 389
for PKI to auth to LDAP. I made the necessary changes and PKI
came up successfully.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>
<http://www.crosschx.com/>
On Fri, May 5, 2017 at 3:19 PM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>
<http://www.crosschx.com/>
On Fri, May 5, 2017 at 3:15 PM, Rob Crittenden

Post by Michael Plemmons
I just realized that I sent the reply directly

to Rob
and not to the

Post by Michael Plemmons
list. My response is inline

Ok, this is actually good news.
I made a similar proposal in another case and I was
completely wrong.
Flo had the user do something and it totally fixed their
auth error, I
just can't remember what it was or find the e-mail
thread. I'm pretty
sure it was this calendar year though.
rob
Do you or Flo know what I could search for in the past
emails to find the answer to the problem?

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com>
<http://www.crosschx.com/>

Post by Michael Plemmons
On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer |

CROSSCHX

Post by Michael Plemmons
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com>
<http://www.crosschx.com/>

Post by Michael Plemmons
On Thu, May 4, 2017 at 9:24 AM, Rob Crittenden

Post by Michael Plemmons
I realized that I was not very clear

in my
statement about

Post by Michael Plemmons
testing with

Post by Michael Plemmons
ldapsearch. I had initially run it

without
logging in with a

Post by Michael Plemmons
DN. I was

Post by Michael Plemmons
just running the local ldapsearch -x

command. I then tested on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt and ipa11.mgmt logging in

with a
full DN for the

Post by Michael Plemmons
admin and

Post by Michael Plemmons
"cn=Directory Manager" from ipa12.mgmt

(broken server) and

Post by Michael Plemmons
ipa11.mgmt

Post by Michael Plemmons
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt and

ipa11.mgmt as a non

Post by Michael Plemmons
root user.

Post by Michael Plemmons
I also ran the command showing a

line count
for the output and

Post by Michael Plemmons
the line

Post by Michael Plemmons
counts for each were the same when

run from
ipa12.mgmt and

Post by Michael Plemmons
ipa11.mgmt.

Post by Michael Plemmons
ldapsearch -LLL -h

ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>> -D "DN" -w PASSWORD -b
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn

Post by Michael Plemmons

Post by Michael Plemmons
ldapsearch -LLL -h

ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>> -D "cn=directory
manager" -w

Post by Michael Plemmons
PASSWORD dn
The CA has its own suffix and replication

agreements. Given the auth

Post by Michael Plemmons
error and recent (5 months) renewal of CA

credentials I'd check

Post by Michael Plemmons
that the
CA agent authentication entries are

correct.

Post by Michael Plemmons
$ ldapsearch -LLL -x -D 'cn=directory

manager'
-W -b

Post by Michael Plemmons
uid=ipara,ou=people,o=ipaca description
The format is 2;serial#,subject,issuer
# certutil -L -d /etc/httpd/alias -n

ipaCert
|grep Serial

Post by Michael Plemmons
The serial # should match that in the

description everywhere.

Post by Michael Plemmons
rob
On the CA (IPA13.MGMT) I ran the ldapsearch

command and see that the

Post by Michael Plemmons
serial number is 7. I then ran the certutil

command on all three

Post by Michael Plemmons
servers and the serial number is 7 as well.
I also ran the ldapsearch command against the

other two servers and

Post by Michael Plemmons
they also showed a serial number of 7.

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps

Engineer |
CROSSCHX

Post by Michael Plemmons

Post by Michael Plemmons
*
614.427.2411
www.crosschx.com

<http://www.crosschx.com> <http://www.crosschx.com>
<http://www.crosschx.com>

Post by Michael Plemmons
<http://www.crosschx.com/>

Post by Michael Plemmons
On Wed, May 3, 2017 at 5:28 PM,

Michael Plemmons

Post by Michael Plemmons

Post by Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6

months ago

Post by Michael Plemmons

Post by Michael Plemmons
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11 and

ipa12. ipa11 and

Post by Michael Plemmons
ipa12 do not

Post by Michael Plemmons
have agreements between each other.
It appears that either

ipa12.mgmt lost
some level of its

Post by Michael Plemmons
replication

Post by Michael Plemmons
agreement with ipa13. I saw

some level
because users /

Post by Michael Plemmons
hosts were

Post by Michael Plemmons
replicated between all systems

but we
started seeing DNS

Post by Michael Plemmons
was not

Post by Michael Plemmons
resolving properly from ipa12.

I do not
know when this

Post by Michael Plemmons
started.

Post by Michael Plemmons
When looking at replication

agreements
on ipa12 I did not

Post by Michael Plemmons
see any

Post by Michael Plemmons
agreement with ipa13.
When I run ipa-replica-manage

list all
three hosts show

Post by Michael Plemmons
has master.

Post by Michael Plemmons
When I run ipa-replica-manage

ipa11.mgmt
I see ipa13.mgmt

Post by Michael Plemmons
is a replica.

Post by Michael Plemmons
When I run ipa-replica-manage

ipa12.mgmt
nothing returned.

Post by Michael Plemmons

Post by Michael Plemmons
I ran ipa-replica-manage connect

--cacert=/etc/ipa/ca.crt

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>>

Post by Michael Plemmons

Post by Michael Plemmons
ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>> on ipa12.mgmt

Post by Michael Plemmons

Post by Michael Plemmons
I then ran the following
ipa-replica-manage force-sync --from

ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>>

Post by Michael Plemmons

Post by Michael Plemmons
ipa-replica-manage re-initialize

--from

Post by Michael Plemmons
ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>>

Post by Michael Plemmons

Post by Michael Plemmons
I was still seeing bad DNS

returns when
dig'ing against

Post by Michael Plemmons
ipa12.mgmt.

Post by Michael Plemmons
I was able to create user and DNS

records and see the

Post by Michael Plemmons
information

Post by Michael Plemmons
replicated properly across all

three nodes.

Post by Michael Plemmons

Post by Michael Plemmons
I then ran ipactl stop on

ipa12.mgmt and
then ipactl start on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt because I wanted to

make sure
everything was

Post by Michael Plemmons
running

Post by Michael Plemmons
fresh after the changes above.

While
IPA was staring up (DNS

Post by Michael Plemmons

Post by Michael Plemmons
started) we were able to see

valid DNS
queries returned but

Post by Michael Plemmons

Post by Michael Plemmons
pki-tomcat would not start.
I am not sure what I need to do

in order
to get this

Post by Michael Plemmons
working. I

Post by Michael Plemmons
have included the output of

certutil and
getcert below

Post by Michael Plemmons
from all

Post by Michael Plemmons
three servers as well as the debug

output for pki.

Post by Michael Plemmons

Post by Michael Plemmons
While the IPA system is coming

up I am
able to

Post by Michael Plemmons
successfully run

Post by Michael Plemmons
ldapsearch -x as the root user

and see
results. I am also

Post by Michael Plemmons
able to

Post by Michael Plemmons
login with the "cn=Directory

Manager"
account and see results.

Post by Michael Plemmons

Post by Michael Plemmons
The debug log shows the

following error.
============================================
===== DEBUG

Post by Michael Plemmons

Post by Michael Plemmons
SUBSYSTEM INITIALIZED =======

============================================

Post by Michael Plemmons
restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?

/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

Post by Michael Plemmons
about to

Post by Michael Plemmons
look for cert for auto-shutdown

support:auditSigningCert

Post by Michael Plemmons
cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=debug
initialized debug
initSubsystem id=log

ready to

Post by Michael Plemmons
init id=log

[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
[03/May/2017:21:22:01][localhost-startStop-1]: Creating
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)

Post by Michael Plemmons
restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?

/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

Post by Michael Plemmons
about to

Post by Michael Plemmons
look for cert for auto-shutdown

support:auditSigningCert

Post by Michael Plemmons
cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=log
initialized log
initSubsystem id=jss

ready to

Post by Michael Plemmons
init id=jss

restart at

Post by Michael Plemmons
autoShutdown? false
autoShutdown crumb file path?

/var/lib/pki/pki-tomcat/logs/autoShutdown.crumb

Post by Michael Plemmons
about to

Post by Michael Plemmons
look for cert for auto-shutdown

support:auditSigningCert

Post by Michael Plemmons
cert-pki-ca
found

Post by Michael Plemmons
cert:auditSigningCert cert-pki-ca

done init

Post by Michael Plemmons
id=jss
initialized jss
initSubsystem id=dbs

ready to

Post by Michael Plemmons
init id=dbs

DBSubsystem: init()

Post by Michael Plemmons
mEnableSerialMgmt=true

[03/May/2017:21:22:01][localhost-startStop-1]: Creating

Post by Michael Plemmons

Post by Michael Plemmons
LdapBoundConnFactor(DBSubsystem)
init
LdapBoundConnFactory:doCloning true

LdapAuthInfo: init()
LdapAuthInfo: init begins
LdapAuthInfo: init ends

init: before

Post by Michael Plemmons

Post by Michael Plemmons
makeConnection errorIfDown is true
errorIfDown true

Setting
desired cert

Post by Michael Plemmons

Post by Michael Plemmons
subsystemCert cert-pki-ca

LdapJssSSLSocket: set

Post by Michael Plemmons
client auth cert nickname

subsystemCert
cert-pki-ca
Entering!
returning: null
[03/May/2017:21:22:02][localhost-startStop-1]: SSL

Post by Michael Plemmons
handshake happened

Post by Michael Plemmons
Could not connect to LDAP server

host

Post by Michael Plemmons
ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>> port 636 Error
Authentication failed (48)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:166)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:130)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1169)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1075)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.certsrv.apps.CMS.init(CMS.java:187)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.certsrv.apps.CMS.start(CMS.java:1616)

Post by Michael Plemmons

Post by Michael Plemmons
at

com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)

Post by Michael Plemmons

Post by Michael Plemmons
at

javax.servlet.GenericServlet.init(GenericServlet.java:158)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke0(Native

Post by Michael Plemmons
Method)

Post by Michael Plemmons
at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

Post by Michael Plemmons

Post by Michael Plemmons
at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.lang.reflect.Method.invoke(Method.java:498)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.security.AccessController.doPrivileged(Native

Post by Michael Plemmons
Method)

Post by Michael Plemmons
at

javax.security.auth.Subject.do
<http://javax.security.auth.Subject.do>
<http://javax.security.auth.Subject.do
<http://javax.security.auth.Subject.do>>

Post by Michael Plemmons
<http://javax.security.auth.Subject.do

<http://javax.security.auth.Subject.do>
<http://javax.security.auth.Subject.do
<http://javax.security.auth.Subject.do>>>AsPrivileged(Subject.java:549)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)

Post by Michael Plemmons

Post by Michael Plemmons
at

org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)

Post by Michael Plemmons

Post by Michael Plemmons
at

java.security.AccessController.doPrivileged(Native

[Michael Plemmons]

SOLVED!

Thank you Flo! That did the trick. Once I made the change to the
certificate and restarted the IPA services everything came back up like it
was supposed to.

High five!


*Mike Plemmons | Senior DevOps Engineer | CROSSCHX*
614.427.2411
***@crosschx.com
www.crosschx.com

Post by Florence Blanc-Renaud

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com/>
I have done more searching in my logs and I see the following errors.
This is in the localhost log file /var/lib/pki/pki-tomcat/logs
May 15, 2017 3:08:08 PM
org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
java.lang.NullPointerException
May 15, 2017 3:08:08 PM org.apache.catalina.core.StandardContext
loadOnStartup
SEVERE: Servlet [castart] in web application [/ca] threw load() exception
java.lang.NullPointerException
May 15, 2017 3:08:09 PM
org.apache.catalina.core.StandardHostValve invoke
SEVERE: Exception Processing /ca/admin/ca/getStatus
javax.ws.rs <http://javax.ws.rs>
<http://javax.ws.rs>.ServiceUnavailableException: Subsystem
unavailable
Looking at the debug log it says Authentication failed for port 636.
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init()
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init begins
[15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init ends
[15/May/2017:17:39:25][localhost-startStop-1]: init: before
makeConnection errorIfDown is true
errorIfDown true
subsystemCert cert-pki-ca
[15/May/2017:17:39:25][localhost-startStop-1]: LdapJssSSLSocket: set
client auth cert nickname subsystemCert cert-pki-ca
SSLClientCertificatSelectionCB: Entering!
SSLClientCertificateSelectionCB: returning: null
[15/May/2017:17:39:25][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>> port 636 Error
netscape.ldap.LDAPException: Authentication failed (48)
at
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConne
ction(LdapBoundConnFactory.java:205)
I looked at the validity of the cert it mentions and it is fine.
(root)>getcert status -v -d /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca'
State MONITORING, stuck: no.
I then looked at the ldap errors around the time of this failure and I
am seeing this log entry.
[15/May/2017:17:38:42.063080758 +0000] set_krb5_creds - Could not get
initial credentials for principal
[FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC for
requested realm)
When I perform a klist against that keytab nothing appears out of the
ordinary compared to working IPA servers.
I am not sure what to look at next.
Hi,
you can try the following to manually replay the connection
root$ export LDAPTLS_CACERTDIR=/etc/pki/pki-tomcat/alias
root$ export LDAPTLS_CERT='subsystemCert cert-pki-ca'
The above commands specify the NSSDB containing the user certificate
and its name for SASL-EXTERNAL authentication.
Then note the value obtained below as it will be used for the next
root$ grep internal /etc/pki/pki-tomcat/password.conf
internal=<some value>
root$ ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL
-Q -LLL dn namingcontexts
Please enter pin, password, or pass phrase for security token
'ldap(0)': <<<< here supply the value found above
namingcontexts: cn=changelog
namingcontexts: dc=ipadomain,dc=com
namingcontexts: o=ipaca
So I guess I found my problem.
(root)>ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL -Q
-LLL dn namingcontexts
ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
additional info: TLS error -12195:Peer does not recognize and trust
the CA that issued your certificate.
I looked at our certs in /etc/dirsrv/slapd-IPADOMAIN-COM and found the
following.
IPA12 - problem server
(root)>certutil -L -d /etc/dirsrv/slapd-IPADOMAIN-COM
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,u
IPADOMAIN-COM IPA CA C,,
IPA11/IPA13 - 11 was the master and 13 is the new master
(root)>certutil -L -d /etc/dirsrv/slapd-IPADOMAIN-COM
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,u
IPADOMAIN-COM IPA CA CT,C,C

root$ certutil -M -d /etc/dirsrv/slapd-IPADOMAIN-COM -n 'IPADOMAIN-COM
IPA CA' -t CT,C,C
Flo.

Post by Michael Plemmons
In the LDAP server access log (in
/etc/dirsrv/slapd-IPADOMAIN.COM/access), you should see the
[18/May/2017:13:35:14.822090417 +0200] conn=297 fd=150 slot=150 SSL
connection from xxx to yyy
[18/May/2017:13:35:15.789414017 +0200] conn=297 TLS1.2 128-bit
AES-GCM; client CN=CA Subsystem,O=IPADOMAIN.COM
<http://IPADOMAIN.COM>; issuer CN=Certificate
Authority,O=IPADOMAIN.COM <http://IPADOMAIN.COM>
[18/May/2017:13:35:15.793108509 +0200] conn=297 TLS1.2 client bound
as uid=pkidbuser,ou=people,o=ipaca
[18/May/2017:13:35:15.798101505 +0200] conn=297 op=0 BIND dn=""
method=sasl version=3 mech=EXTERNAL
[18/May/2017:13:35:15.800322076 +0200] conn=297 op=0 RESULT err=0
tag=97 nentries=0 etime=0 dn="uid=pkidbuser,ou=people,o=ipaca"
HTH,
Flo.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>
<http://www.crosschx.com/>
On Wed, May 10, 2017 at 3:35 PM, Michael Plemmons
The PKI service came up successfully but only when it uses BasicAuth
rather than SSL auth. I am not sure about what I need to do in
order to get the auth working over SSL again.
None of the certs are expired when I run getcert list and
ipa-getcert list.
Since the failure is with attempts to login to LDAP over 636. I
have been attempting to auth to LDAP via port 636 and the ldapsearch
is not completing. When looking at packet captures, I see some the
TCP handshake and what appears to be the start of a SSL process and
then everything hangs.
What is the proper method to test performing a ldapsearch over 636?
Also, the CS.cfg shows it wants to auth as cn=Directory Manager. I
can successfully auth with cn=Directory Manager over 389 but I think
I am not performing ldapsearch over 636 correctly.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>
<http://www.crosschx.com/>
On Fri, May 5, 2017 at 3:33 PM, Michael Plemmons
I think I found the email thread. Asking for help with crashed
freeIPA istance. That email pointed to this
link,
https://www.redhat.com/archives/freeipa-users/2017-January/
msg00215.html
<https://www.redhat.com/archives/freeipa-users/2017-January/
msg00215.html>
<https://www.redhat.com/archives/freeipa-users/2017-January/
msg00215.html
<https://www.redhat.com/archives/freeipa-users/2017-January/
msg00215.html>>.
That link talked about changing the CS.cfg file to use port 389
for PKI to auth to LDAP. I made the necessary changes and PKI
came up successfully.
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>
<http://www.crosschx.com/>
On Fri, May 5, 2017 at 3:19 PM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>
<http://www.crosschx.com/>
On Fri, May 5, 2017 at 3:15 PM, Rob Crittenden

Post by Michael Plemmons
I just realized that I sent the reply directly

to Rob
and not to the

Post by Michael Plemmons
list. My response is inline

Ok, this is actually good news.
I made a similar proposal in another case and I was
completely wrong.
Flo had the user do something and it totally fixed their
auth error, I
just can't remember what it was or find the e-mail
thread. I'm pretty
sure it was this calendar year though.
rob
Do you or Flo know what I could search for in the past
emails to find the answer to the problem?

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer |

CROSSCHX

Post by Michael Plemmons
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com>
<http://www.crosschx.com/>

Post by Michael Plemmons
On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons
*Mike Plemmons | Senior DevOps Engineer |

CROSSCHX

Post by Michael Plemmons
*
614.427.2411
www.crosschx.com <http://www.crosschx.com>

<http://www.crosschx.com>
<http://www.crosschx.com/>

Post by Michael Plemmons
On Thu, May 4, 2017 at 9:24 AM, Rob

Crittenden

Post by Michael Plemmons

Post by Michael Plemmons
I realized that I was not very clear

in my
statement about

Post by Michael Plemmons
testing with

Post by Michael Plemmons
ldapsearch. I had initially run it

without
logging in with a

Post by Michael Plemmons
DN. I was

Post by Michael Plemmons
just running the local ldapsearch -x

command. I then tested on

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt and ipa11.mgmt logging in

with a
full DN for the

Post by Michael Plemmons
admin and

Post by Michael Plemmons
"cn=Directory Manager" from ipa12.mgmt

(broken server) and

Post by Michael Plemmons
ipa11.mgmt

Post by Michael Plemmons
and both ldapsearch command succeeded.
I ran the following from ipa12.mgmt

and
ipa11.mgmt as a non

Post by Michael Plemmons
root user.

Post by Michael Plemmons
I also ran the command showing a

line count
for the output and

Post by Michael Plemmons
the line

Post by Michael Plemmons
counts for each were the same when

run from
ipa12.mgmt and

Post by Michael Plemmons
ipa11.mgmt.

Post by Michael Plemmons
ldapsearch -LLL -h

ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>> -D "DN" -w PASSWORD -b
"cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn

Post by Michael Plemmons

Post by Michael Plemmons
ldapsearch -LLL -h

ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>> -D "cn=directory
manager" -w

Post by Michael Plemmons
PASSWORD dn
The CA has its own suffix and

replication
agreements. Given the auth

Post by Michael Plemmons
error and recent (5 months) renewal of

CA
credentials I'd check

Post by Michael Plemmons
that the
CA agent authentication entries are

correct.

Post by Michael Plemmons
$ ldapsearch -LLL -x -D 'cn=directory

manager'
-W -b

Post by Michael Plemmons
uid=ipara,ou=people,o=ipaca description
The format is 2;serial#,subject,issuer
# certutil -L -d /etc/httpd/alias -n

ipaCert
|grep Serial

Post by Michael Plemmons
The serial # should match that in the

description everywhere.

Post by Michael Plemmons
rob
On the CA (IPA13.MGMT) I ran the ldapsearch

command and see that the

Post by Michael Plemmons
serial number is 7. I then ran the certutil

command on all three

Post by Michael Plemmons
servers and the serial number is 7 as well.
I also ran the ldapsearch command against

the
other two servers and

Post by Michael Plemmons
they also showed a serial number of 7.

Post by Michael Plemmons
*Mike Plemmons | Senior DevOps

Engineer |
CROSSCHX

Post by Michael Plemmons

Post by Michael Plemmons
*
614.427.2411
www.crosschx.com

<http://www.crosschx.com> <http://www.crosschx.com>
<http://www.crosschx.com>

Post by Michael Plemmons
<http://www.crosschx.com/>

Post by Michael Plemmons
On Wed, May 3, 2017 at 5:28 PM,

Michael Plemmons

Post by Michael Plemmons

Post by Michael Plemmons
I have a three node IPA cluster.
ipa11.mgmt - was a master over 6

months ago

Post by Michael Plemmons

Post by Michael Plemmons
ipa13.mgmt - current master
ipa12.mgmt
ipa13 has agreements with ipa11

and
ipa12. ipa11 and

Post by Michael Plemmons
ipa12 do not

Post by Michael Plemmons
have agreements between each

other.

Post by Michael Plemmons

Post by Michael Plemmons
It appears that either

ipa12.mgmt lost
some level of its

Post by Michael Plemmons
replication

Post by Michael Plemmons
agreement with ipa13. I saw

some level
because users /

Post by Michael Plemmons
hosts were

Post by Michael Plemmons
replicated between all systems

but we
started seeing DNS

Post by Michael Plemmons
was not

Post by Michael Plemmons
resolving properly from ipa12.

I do not
know when this

Post by Michael Plemmons
started.

Post by Michael Plemmons
When looking at replication

agreements
on ipa12 I did not

Post by Michael Plemmons
see any

Post by Michael Plemmons
agreement with ipa13.
When I run ipa-replica-manage

list all
three hosts show

Post by Michael Plemmons
has master.

Post by Michael Plemmons
When I run ipa-replica-manage

ipa11.mgmt
I see ipa13.mgmt

Post by Michael Plemmons
is a replica.

Post by Michael Plemmons
When I run ipa-replica-manage

ipa12.mgmt
nothing returned.

Post by Michael Plemmons

Post by Michael Plemmons
I ran ipa-replica-manage connect

--cacert=/etc/ipa/ca.crt

Post by Michael Plemmons

Post by Michael Plemmons
ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>

Post by Michael Plemmons
<http://ipa12.mgmt.crosschx.com

<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>
<http://ipa12.mgmt.crosschx.com
<http://ipa12.mgmt.crosschx.com>>>>

Post by Michael Plemmons

Post by Michael Plemmons
ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>> on ipa12.mgmt

Post by Michael Plemmons

Post by Michael Plemmons
I then ran the following
ipa-replica-manage force-sync

--from

Post by Michael Plemmons
ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>>

Post by Michael Plemmons

Post by Michael Plemmons
ipa-replica-manage re-initialize

--from

Post by Michael Plemmons
ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>

Post by Michael Plemmons

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>

Post by Michael Plemmons
<http://ipa13.mgmt.crosschx.com

<http://ipa13.mgmt.crosschx.com>
<http://ipa13.mgmt.crosschx.com
<http://ipa13.mgmt.crosschx.com>>>>

Post by Michael Plemmons

Post by Michael Plemmons
I was still seeing bad DNS

returns when
dig'ing against

Post by Michael Plemmons
ipa12.mgmt.

Post by Michael Plemmons
I was able to create user and DNS

records and see the

Post by Michael Plemmons
information

Post by Michael Plemmons
replicated properly across all

three nodes.

Post by Michael Plemmons

Post by Michael Plemmons
I then ran ipactl stop on

ipa12.mgmt and
then ipactl start on