Starting services and populating new servers with users, gateways, devices, etc.

In previous posts we’ve demonstrated how to create a sipXcom AMI from a base CentOS template, how to install and configure sipXcom on that template and how to add Route 53 entries that points to that machine.

It’s time to bring all of this together and automate creation of a configured machine.

Step 1. Add superadmin password from command line interface

Command that we will use will be an postgres insert that will look like:
SIPXCONFIG=# INSERT INTO users VALUES (nextval(‘user_seq’), ‘Mihai’, ‘123’,’321′,’Costache’, ‘MirceaMihai’);

Note: “nextval(‘user_seq’)” – This parameter is a must to increment the primary key used by “users” table.

We will run this command in devops style from ansible without first connecting to psql by adding a stored procedure and running it from ansible shell module. This will add superadmin permissions to superadmin user.

Below is the stored procedure content:

$ cat files/superadmin.sql
           CREATE FUNCTION create_superadmin() RETURNS integer AS $$

           DECLARE
	         valueStorageId integer;  
           BEGIN
             select nextval('storage_seq') into valueStorageId;
             INSERT INTO value_storage VALUES (valueStorageId);
             INSERT INTO setting_value VALUES (valueStorageId, 'ENABLE','permission/application/superadmin');
             INSERT INTO users VALUES (nextval('user_seq'), NULL, 'Ezuce123', 'Ezuce123', NULL, 'superadmin', valueStorageId, NULL, false, 
             NULL, 'C', 'Ezuce123', true);
	         RETURN 0;
             END;
             $$ LANGUAGE plpgsql;
             select create_superadmin();

To execute this sql script we will add following lines in our playbook:

  – name: copy superadmin.sql function to remote machine

               copy: src=”files/superadmin.sql”  dest=”/tmp/superadmin.sql”

  – name: Execute psql script 

               shell: psql -U postgres SIPXCONFIG -f /tmp/superadmin.sql

  – name: Start services

               script: files/start_services.sh


Step 2. Start services on remote server

In order to start services on sipXcom server we need to create a script that will insert which services we want to start and then restart sipxconfig. You can check script content below

cat files/start_services.sh
#!/usr/bin/env bash

psql -U postgres SIPXCONFIG << EOF
INSERT INTO feature_global (feature_id) VALUES (‘natTraversal’);
INSERT INTO feature_global (feature_id) VALUES (‘snmp’);
INSERT INTO feature_global (feature_id) VALUES (‘sipxlogwatcher’);
INSERT INTO feature_global (feature_id) VALUES (‘mail’);
INSERT INTO feature_global (feature_id) VALUES (‘ntpd’);
INSERT INTO feature_global (feature_id) VALUES (‘fail2ban’);
INSERT INTO feature_global (feature_id) VALUES (‘alarms’);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘sipxdns’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘systemaudit’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘elasticsearch’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘apache’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘mongo’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘phonelog’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘ftp’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘provision’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘tftp’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘freeSwitch’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘ivr’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘recording’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘redis’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘moh’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘park’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘registrar’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘sipxcdr’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘sipxsqa’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘mwi’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘proxy’, 1);
INSERT INTO feature_local (feature_id, location_id) VALUES (‘conference’, 1);

EOF

service sipxconfig restart

Step 3. Use Python to populate server with REST API

To see all exposed REST API’s you can check this URL:
https://openuc.ezuce.com/sipxconfig/api/?_wadl

First we need to get json files with users, phones, gateways and dialplan to use. To do so I’ve used an already created machine.

To “pretty” visualize the output you need to run:

      $ curl --insecure -XGET -u superadmin:Ezuce123 https://10.3.0.150/sipxconfig/api/rules/ -H "Content-type: application/json" | python 
      -m json.tool

You can save this file in the required json format by running

      $ curl --insecure -XGET -u superadmin:Ezuce123 https://10.3.0.150/sipxconfig/api/rules/ -H "Content-type: application/json" > 
      users.json

Do the same for the rest of the API

Next we will create a python script that will iterate through the above json file, create an object and then POST it to API url:

$ cat files/add_users_API.py
           #!/usr/bin/env python
           import json, requests, yaml

           output_json =  yaml.load(open('/var/log/users.json'))
           headers = {'content-type': 'application/json'}
           for i in output_json:
   	 
          for k in output_json[i]:
                    	data=json.dumps(k,ensure_ascii=False)
                    	new_data=data.encode("utf-8")
                  	 
                    	r = requests.post("https://superadmin:Ezuce123@localhost/sipxconfig/api/users/", 
                        data=new_data,verify=False,headers=headers)
               	

Step 4. Bringing it all together

The following is a one step playbook that will be executed to start a configured and populated machine is presented next:

$ cat populated_sipxcom_final.yml
         ---
         - hosts: localhost
           connection: local
           remote_user: devops
           gather_facts: no
           vars_files:
           - files/awscreds.yml
           - files/info_hosted.yml
           tasks:
           - name: Basic provisioning of EC2 instance
	           ec2:
  	           aws_access_key: "{{ aws_id }}"
  	           aws_secret_key: "{{ aws_key }}"
  	           region: "{{ aws_region }}"
  	           image: "{{ sipxcom_ami }}"
  	           instance_type: "{{ large_instance }}"
  	           key_name: "{{ ssh_keyname }}"
  	           count: 1
  	           state: present
  	           group_id: "{{ secgroup_id }}"
  	           wait: yes
  	           vpc_subnet_id: subnet-29a32841
  	           assign_public_ip: yes
  	           instance_tags:
    	           Name: uc1.mihai.ezuce.ro
	           register: ec2info
           - name: Print the results
	           debug: var=ec2info
           - name: Add all instance public IPs to host group
	           add_host: hostname={{ ec2info.instances[0].public_ip  }}
          	   groups=ec2hosts
          	   ansible_ssh_private_key_file=~/.ssh/devops-key.pem

         	 
           - name: Add Route 53 entries for newly created EC2 Instance
	           route53:
  	           command: create
  	           zone: ezuce.ro
  	           overwrite: yes
  	           record: "{{ net_host }}.{{ net_domain }}"
  	           type: A
  	           ttl: 7200
  	           value: "{{ ec2info.instances[0].public_ip }}"
         	 
           - name: Add TCP SIP  SRV
	           route53:
  	           command: create
  	           zone: ezuce.ro
  	           overwrite: yes
  	           record: "_sip._tcp.{{ net_domain }}."
  	           type: SRV
  	           value: "10 10 5060 {{ net_host }}.{{ net_domain }}"
	 
         	 
           - name: Add UDP SIP  SRV
	           route53:
  	           command: create
  	           zone: ezuce.ro
  	           overwrite: yes
  	           record: "_sip._udp.{{ net_domain }}."
  	           type: SRV
  	           value: "10 10 5060 {{ net_host }}.{{ net_domain }}"
 
           - name: Wait for SSH to come up
	           remote_user: root
	           wait_for: host={{ ec2info.instances[0].public_ip  }} port=22 delay=10 timeout=320 state=started


           - hosts: ec2hosts
             vars_files:
  	             - files/awscreds.yml
  	             - files/info_hosted.yml
             remote_user: root
             tasks:
             - name: Configure sipxcom
             shell: sipxecs-setup --noui --sip_domain "{{ sip_domain }}" --sip_realm "{{ sip_realm }}" --net_domain "{{ net_domain }}" 
            --net_host "{{ net_host }}"

 
            - name: copy superadmin.sql function to remote machine
              copy: src="files/superadmin.sql"  dest="/var/log/superadmin.sql"
            - name: Execute psql script  
              shell: psql -U postgres SIPXCONFIG -f /var/log/superadmin.sql
            - name: Start services
              script: files/start_services.sh
            - name: Install cURL for API
              yum: name=curl state=latest
            - name: Copy users.json
              copy: src="files/users.json" dest="/var/log/users.json"
            - name: Copy  phones.json
              copy: src="files/phones.json" dest="/var/log/phones.json"
            - name: Copy gateways.json
                    copy: src="files/gateways.json" dest="/var/log/gateways.json"
            - name: Copy dialplan.json
                    copy: src="files/dialplan.json" dest="/var/log/dialplan.json"
            - name: add users with API
                    script: add_users_API.py
            - name: add phones with API
                    script: add_phones_API.py
            - name: add gateways with API
                    script: add_gateways_API.py
            - name: add dialplan with API

If you followed this series or articles these commands should be familiar to you. We have copied *.json files to EC2 /var/log path, after that we have executed *.py scripts that are momentarily hardcoded to execute using *.json files found in /var/log path.

Now we’re at the end of this series. You should have a fully functional sipXcom server just by running:

$ ansible-playbook populated_sipxcom_final.yml

———————————————————————————-

TASK [add users with API] ******************************************************
changed: [52.211.154.83]

TASK [add phones with API] *****************************************************
changed: [52.211.154.83]

TASK [add gateways with API] ***************************************************
changed: [52.211.154.83]

TASK [add dialplan with API] ***************************************************
changed: [52.211.154.83]

PLAY RECAP *********************************************************************
52.211.154.83 : ok=14 changed=13 unreachable=0 failed=0
localhost : ok=7 changed=3 unreachable=0 failed=0

Complete Proof of Concept files can be downloaded from here:
https://github.com/Mihai-CMM/Ansible_Aws_SipXcom.git

To see the previous blogs in this AWS Ansible sipXcom Blog Series click below:

Part 1: Learn how to Provision a minimal CentOS machine with Ansible-AWS, used to create our own sipXcom based AMI

Part 2: Learn how to create a sipXcom-based AMI

Part 3: Create an EC2 instance from sipXcom template and configure newly launched server

Part 4: Add AWS Route 53 entries to match a newly created server.

This post completes the AWS Ansible sipXcom blog series! Be sure to look out for more helpful blogs coming soon!