Discussion:
[Freeipa-users] Export user and host list to a csv or text file
Sanju A
2014-05-23 04:42:51 UTC
Permalink
Dear All,

Is there any command to export the user and host list to a csv or text format


Regards
Sanju Abraham
___________
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
Bret Wortman
2014-05-23 11:40:07 UTC
Permalink
Yes, though it might be a bit more data than you're expecting.

Here's what we did to get the details out of a server (and import them
into another). I'm sure there's a more elegant solution, but this worked
for us. Also note that we didn't use all the data this export script
generated, but felt it was better to have it than to not.

EXPORT:

#!/bin/sh
#
# Generate latest ipa config files for possible re-import later.
#
# (C) 2014, The Damascus Group
#

CONFIGDIR=/opt/ipa_config

[ ! -d $CONFIGDIR ] && mkdir $CONFIGDIR
pushd $CONFIGDIR

ipa dnszone-find --all > dnszone.txt
grep 'Zone name' dnszone.txt | awk '{print $3}' | sed 's/\r//' > zones.txt
for line in $(cat zones.txt); do
fn=$(echo $line | sed 's/\.in-addr\.arpa\.//')
echo "For zone $line -> dnsrecord-$fn.txt"
ipa dnsrecord-find $line --sizelimit=99999 --all --structured >
dnsrecord-${fn}.txt
done
ipa user-find --all > users.txt
ipa host-find --sizelimit=99999 --all > hosts.txt
ipa policy-find --all > policy.txt
ipa sudorule-find --all > sudorule.txt
ipa sudocmdgroup-find --all > sudocmdgroup.txt
ipa sudocmd-find --all > sudocmd.txt
ipa role-find --all > roles.txt
ipa pwpolicy-find --all > pwpolicy.txt
ipa privilege-find --all > privilege.txt
ipa permission-find --all > permission.txt
ipa netgroup-find --all > netgroup.txt
ipa usergroup-find --all > usergroup.txt
ipa idrange-find --all > idrange.txt
ipa hostgroup-find --all > hostgroup.txt
ipahbacrule-find --all > hbacrule.txt
ipa hbacsvc-find --all > hbacsvc.txt
ipa group-find --all > group.txt
ipa cert-find --all > cert.txt
ipa automember-find --type=group --all > automember-group.txt
ipa automember-find --type=hostgroup --all > automember-hostgroup.txt
popd
------cut-------

Then, for example, you can import these into a new IPA server using
something like these:

#!/bin/bash
#
# parse_hosts
#
# (C) 2014, The Damascus Group
#

FN=$1
OTP=MyOnetimePassword

RE_HOSTNAME="Host name:\s+(.*)$"

name=""

while read line; do
if [[ $line =~ "$name" ]]; then
if [[ -n "$name" ]]; then
echo "Adding $name"
ipa host-add $name --password $OTP --force
fi
name=${BASH_REMATCH[1]}
fi
done < $FN
echo "Adding $name"
ipa host-add $name --password $OTP --force
-------cut----------

And this for users:

#!/bin/bash
#
# parse_users
#
# (C) 2014, The Damascus Group

FN=$1

RE_DN="dn:\s+(.*)$"
RE_LOGIN="User login:\s+(.*)$"
RE_LAST="Last name:\s+(.*)$"
RE_FIRST="First name:\s+(.*)$"
RE_CN="Full name:\s+(.*)$"
RE_DISPLAYNAME="Display name:\s+(.*)$"
RE_INITIALS="Initials:\s+(.*)$"
RE_SHELL="Login shell:\s+(.*)$"
RE_HOMEDIR="Home directory:\s+(.*)$"
RE_PRINCIPAL="Kerberos principal:\s+(.*)$"
RE_EMAIL="Email address:\s+(.*)$"
RE_SSHPUBKEY="SSH public key:\s+(.*)$"
RE_UID="UID:\s+(.*)$"
RE_GID="GID:\s+(.*)$"

login=""
last=""
first=""
cn=""
displayname=""
initials=""
shell=""
homedir=""
prinicpal=""
email=""
sshpubkey=""
uid=""
gid=""

while read line; do
if [[ $line =~ $RE_DN ]]; then
ipa user-add $login \
--last=$last \
--first=$first \
--cn="$cn" \
--displayname="$displayname" \
--initials=$initials \
--shell=$shell \
--homedir=$homedir \
--principal=$principal \
--email=$email \
--sshpubkey="$sshpubkey" \
--uid=$uid \
--gid=$gid
login=""
last=""
first=""
cn=""
displayname=""
initials=""
shell=""
homedir=""
prinicpal=""
email=""
sshpubkey=""
uid=""
gid=""
fi
if [[ $line =~ $RE_LOGIN ]]; then
login=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_LAST ]]; then
last=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_FIRST ]]; then
first=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_CN ]]; then
cn=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_DISPLAYNAME ]]; then
displayname=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_INITIALS ]]; then
initials=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_SHELL ]]; then
shell=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_HOMEDIR ]]; then
homedir=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_PRINCIPAL ]]; then
principal=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_EMAIL ]]; then
email=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_SSHPUBKEY ]]; then
sshpubkey1=${BASH_REMATCH[1]}
read sshpubkey2
read sshpubkey3
sshpubkey="$sshpubkey1 $sshpubkey2 $sshpubkey3"
fi
if [[ $line =~ $RE_UID ]]; then
uid=${BASH_REMATCH[1]}
fi
if [[ $line =~ $RE_GID ]]; then
gid=${BASH_REMATCH[1]}
fi
done < $FN
ipa user-add $login \
--last=$last \
--first=$first \
--cn="$cn" \
--displayname="$displayname" \
--initials=$initials \
--shell=$shell \
--homedir=$homedir \
--principal=$principal \
--email=$email \
--sshpubkey="$sshpubkey" \
--uid=$uid \
--gid=$gid
---------cut----------

If there's any interest, I can toss these scripts plus a handful of
other parsers for things like DNS, hbac and sudo into a github project.
Unless someone points out a compelling reason to not do things this way.


Bret
Post by Sanju A
Dear All,
Is there any command to export the user and host list to a csv or text format
Regards
Sanju Abraham
___________
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
_______________________________________________
Freeipa-users mailing list
https://www.redhat.com/mailman/listinfo/freeipa-users
Martin Kosek
2014-05-23 11:54:35 UTC
Permalink
Post by Sanju A
Dear All,
Is there any command to export the user and host list to a csv or text format
There is no such command out of the shelf, I would personally just write a
short Python script to export the hosts (or anything else) in a format I need.

Example for host:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#!/usr/bin/python2

from ipalib import api
api.bootstrap(context='exporter', debug=False)
api.finalize()
api.Backend.xmlclient.connect()

hosts = api.Command['host_find']()['result']

for host in hosts:
print host['fqdn'][0]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This will print one host for each new line.

Martin
Bret Wortman
2014-05-23 12:02:04 UTC
Permalink
Is the Python API documented anywhere? I've looked around without success.
Post by Martin Kosek
Post by Sanju A
Dear All,
Is there any command to export the user and host list to a csv or text format
There is no such command out of the shelf, I would personally just write a
short Python script to export the hosts (or anything else) in a format I need.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#!/usr/bin/python2
from ipalib import api
api.bootstrap(context='exporter', debug=False)
api.finalize()
api.Backend.xmlclient.connect()
hosts = api.Command['host_find']()['result']
print host['fqdn'][0]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This will print one host for each new line.
Martin
_______________________________________________
Freeipa-users mailing list
https://www.redhat.com/mailman/listinfo/freeipa-users
Petr Vobornik
2014-05-23 12:33:16 UTC
Permalink
Post by Bret Wortman
Is the Python API documented anywhere? I've looked around without success.
Not yet.

For now, you can use IPA CLI for inspection:

CLI commands are basically API commands, where `_` is replaced by `-`.

List objects:
`ipa help topics`

List object commands:
`ipa help $object`, e.g., `ipa help user`

List command CLI options and parameters:
`ipa $command --help`, e.g., `ipa user-mod --help`

Map command params and options names to API option names:
`ipa show-mappings $command`, e.g., `ipa show-mappings user-add`

More can be read from code or by observing Web UI communication in
browser developer tools - network tab.


Then the python syntax is ~
args = ['arg1', 'arg2']
options = dict(option1="foo", option2="bar")
api.Command['command_name'](*args, **options)

HTH
Post by Bret Wortman
Post by Martin Kosek
Post by Sanju A
Dear All,
Is there any command to export the user and host list to a csv or text format
There is no such command out of the shelf, I would personally just write a
short Python script to export the hosts (or anything else) in a format I need.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#!/usr/bin/python2
from ipalib import api
api.bootstrap(context='exporter', debug=False)
api.finalize()
api.Backend.xmlclient.connect()
hosts = api.Command['host_find']()['result']
print host['fqdn'][0]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This will print one host for each new line.
Martin
--
Petr Vobornik
Chris Swingler
2014-05-23 12:39:12 UTC
Permalink
Another alternative is to use Apache Directory Studio; it can dump most objects out into a CSV, and you should be able to filter out only the data you want.
Post by Petr Vobornik
Post by Bret Wortman
Is the Python API documented anywhere? I've looked around without success.
Not yet.
CLI commands are basically API commands, where `_` is replaced by `-`.
`ipa help topics`
`ipa help $object`, e.g., `ipa help user`
`ipa $command --help`, e.g., `ipa user-mod --help`
`ipa show-mappings $command`, e.g., `ipa show-mappings user-add`
More can be read from code or by observing Web UI communication in browser developer tools - network tab.
Then the python syntax is ~
args = ['arg1', 'arg2']
options = dict(option1="foo", option2="bar")
api.Command['command_name'](*args, **options)
HTH
Post by Bret Wortman
Post by Martin Kosek
Post by Sanju A
Dear All,
Is there any command to export the user and host list to a csv or text format
There is no such command out of the shelf, I would personally just write a
short Python script to export the hosts (or anything else) in a format I need.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#!/usr/bin/python2
from ipalib import api
api.bootstrap(context='exporter', debug=False)
api.finalize()
api.Backend.xmlclient.connect()
hosts = api.Command['host_find']()['result']
print host['fqdn'][0]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This will print one host for each new line.
Martin
--
Petr Vobornik
_______________________________________________
Freeipa-users mailing list
https://www.redhat.com/mailman/listinfo/freeipa-users
Martin Kosek
2014-05-23 13:49:07 UTC
Permalink
Right, that's a good suggestion and should work in many use cases.

You will just miss attributes or modifications done inside FreeIPA server
framework plugins (e.g. conversion of DNS IDN name from punycode to unicode).

Martin
Post by Chris Swingler
Another alternative is to use Apache Directory Studio; it can dump most objects out into a CSV, and you should be able to filter out only the data you want.
Post by Petr Vobornik
Post by Bret Wortman
Is the Python API documented anywhere? I've looked around without success.
Not yet.
CLI commands are basically API commands, where `_` is replaced by `-`.
`ipa help topics`
`ipa help $object`, e.g., `ipa help user`
`ipa $command --help`, e.g., `ipa user-mod --help`
`ipa show-mappings $command`, e.g., `ipa show-mappings user-add`
More can be read from code or by observing Web UI communication in browser developer tools - network tab.
Then the python syntax is ~
args = ['arg1', 'arg2']
options = dict(option1="foo", option2="bar")
api.Command['command_name'](*args, **options)
HTH
Post by Bret Wortman
Post by Martin Kosek
Post by Sanju A
Dear All,
Is there any command to export the user and host list to a csv or text format
There is no such command out of the shelf, I would personally just write a
short Python script to export the hosts (or anything else) in a format I need.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#!/usr/bin/python2
from ipalib import api
api.bootstrap(context='exporter', debug=False)
api.finalize()
api.Backend.xmlclient.connect()
hosts = api.Command['host_find']()['result']
print host['fqdn'][0]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This will print one host for each new line.
Martin
--
Petr Vobornik
_______________________________________________
Freeipa-users mailing list
https://www.redhat.com/mailman/listinfo/freeipa-users
_______________________________________________
Freeipa-users mailing list
https://www.redhat.com/mailman/listinfo/freeipa-users
Continue reading on narkive:
Loading...