Showing posts with label mitaka. Show all posts
Showing posts with label mitaka. Show all posts

Monday, December 12, 2016

Deploying an very basic instance on OpenStack Mitaka using a simple heat orchestration template and associate a Floating IP to the simple stack

Deploying an very basic instance on OpenStack using a simple heat template


The below is a simple sample template which we will use
=========================================================
heat_template_version: 2013-05-23

description: Simple template to deploy a single compute instance

parameters:
  image:
    type: string
    label: Image name or ID
    description: Image to be used for compute instance
    default: cirros-0.3.3-x86_64
  flavor:
    type: string
    label: Flavor
    description: Type of instance (flavor) to be used
    default: m1.small
  key:
    type: string
    label: Key name
    description: Name of key-pair to be used for compute instance
    default: my_key
  private_network:
    type: string
    label: Private network name or ID
    description: Network to attach instance to.
    default: private-net

resources:
  my_instance:
    type: OS::Nova::Server
    properties:
      image: { get_param: image }
      flavor: { get_param: flavor }
      key_name: { get_param: key }
      networks:
        - network: { get_param: private_network }
      user_data: |
        #!/bin/sh
        echo "Hello, World!"
      user_data_format: RAW

outputs:
  instance_name:
    description: Name of the instance
    value: { get_attr: [my_instance, name] }
  instance_ip:
    description: IP address of the instance
    value: { get_attr: [my_instance, first_address] }



The above file has to be copied in the form of an YAML file


We will Need to specify the above in the heat template YAML file: network, instance, flavor, security keys also the instance name

Lets get the information on the resources from the existing OpenStack Setup


Source the required credentials


[root@controller ~]# . keystonerc_admin

See the available flavors
==========================
[root@controller ~(keystone_admin)]# openstack flavor list
+----+-----------+-------+------+-----------+-------+-----------+
| ID | Name      |   RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+-----------+-------+------+-----------+-------+-----------+
| 1  | m1.tiny   |   512 |    1 |         0 |     1 | True      |
| 2  | m1.small  |  2048 |   20 |         0 |     1 | True      |
| 3  | m1.medium |  4096 |   40 |         0 |     2 | True      |
| 4  | m1.large  |  8192 |   80 |         0 |     4 | True      |
| 5  | m1.xlarge | 16384 |  160 |         0 |     8 | True      |
+----+-----------+-------+------+-----------+-------+-----------+


See the available images
========================
[root@controller ~(keystone_admin)]# openstack image list
+--------------------------------------+--------+--------+
| ID                                   | Name   | Status |
+--------------------------------------+--------+--------+
| 44bb15d9-a970-40c4-8b77-a7e71b170659 | Cirros | active |
+--------------------------------------+--------+--------+
[root@controller ~(keystone_admin)]#


See the available networks in neutron
=====================================
[root@controller ~(keystone_admin)]# neutron net-list
+--------------------------------------+-----------+-------------------------------------------------------+
| id                                   | name      | subnets                                               |
+--------------------------------------+-----------+-------------------------------------------------------+
| 60ca0e7b-3c40-482c-92d1-3b4da4126e9d | testnet1  | 8fd08288-a83b-44ee-ac04-f8b88fc69628 172.16.15.0/24   |
| 8649a5a8-c137-444d-950d-e0c0569e3ee4 | internal1 | bd5b5a96-eaf5-45a8-93e5-6be837e61414 172.16.16.0/24   |
| 001ac29b-1f5f-4498-a44b-de0f809d2322 | external  | cf1da694-4a75-4768-b758-859dc750a02d 192.168.205.0/24 |
+--------------------------------------+-----------+-------------------------------------------------------+


See the available subnets in the neutron
========================================
[root@controller ~(keystone_admin)]# neutron subnet-list
+--------------------------------------+--------------+------------------+--------------------------------------------------------+
| id                                   | name         | cidr             | allocation_pools                                       |
+--------------------------------------+--------------+------------------+--------------------------------------------------------+
| 8fd08288-a83b-44ee-ac04-f8b88fc69628 | testsub      | 172.16.15.0/24   | {"start": "172.16.15.2", "end": "172.16.15.254"}       |
| bd5b5a96-eaf5-45a8-93e5-6be837e61414 | internal1sub | 172.16.16.0/24   | {"start": "172.16.16.2", "end": "172.16.16.254"}       |
| cf1da694-4a75-4768-b758-859dc750a02d | extsub       | 192.168.205.0/24 | {"start": "192.168.205.101", "end": "192.168.205.254"} |
+--------------------------------------+--------------+------------------+--------------------------------------------------------+

See the available keypairs for the project
==========================================
[root@controller ~(keystone_admin)]# openstack keypair list
+------+-------------------------------------------------+
| Name | Fingerprint                                     |
+------+-------------------------------------------------+
| key1 | 2f:40:44:36:1d:b9:ab:cd:b6:57:24:c7:70:dc:8c:f7 |
+------+-------------------------------------------------+
[root@controller ~(keystone_admin)]#


Lets edit the YAML file which will look like below now

heat_template_version: 2013-05-23

description: Simple template to deploy a single compute instance

parameters:
  image:
    type: string
    label: Image name or ID
    description: Image to be used for compute instance
    default: Cirros
  flavor:
    type: string
    label: Flavor
    description: Type of instance (flavor) to be used
    default: m1.tiny
  key:
    type: string
    label: Key name
    description: Name of key-pair to be used for compute instance
    default: key1
  private_network:
    type: string
    label: Private network name or ID
    description: Network to attach instance to.
    default: internal1

resources:
  my_instance:
    type: OS::Nova::Server
    properties:
      image: { get_param: image }
      flavor: { get_param: flavor }
      key_name: { get_param: key }
      networks:
        - network: { get_param: private_network }
      user_data: |
        #!/bin/sh
        echo "Hello, World!"
      user_data_format: RAW

outputs:
  instance_name:
    description: Name of the instance
    value: { get_attr: [my_instance, name] }
  instance_ip:
    description: IP address of the instance
    value: { get_attr: [my_instance, first_address] }




Lets copy this YAML file in the OpenStack server as the file heatstack1.yaml


Create the stack syntax:

heat stack-create <Name of the Stack> -f <file containing YAML information of the stack>

heat stack-create stack1 -f heatstack1.yaml


Create the stack using the YAML file
====================================

[root@controller ~(keystone_admin)]# heat stack-create stack1 -f heatstack1.yaml
+--------------------------------------+------------+--------------------+---------------------+--------------+
| id                                   | stack_name | stack_status       | creation_time       | updated_time |
+--------------------------------------+------------+--------------------+---------------------+--------------+
| 5ab8fe95-7ea3-4530-86de-facae9f52c71 | stack1     | CREATE_IN_PROGRESS | 2016-12-07T06:31:07 | None         |
+--------------------------------------+------------+--------------------+---------------------+--------------+
[root@controller ~(keystone_admin)]#


See the list of the stacks
==========================
[root@controller ~(keystone_admin)]# heat stack-list
+--------------------------------------+------------+-----------------+---------------------+--------------+
| id                                   | stack_name | stack_status    | creation_time       | updated_time |
+--------------------------------------+------------+-----------------+---------------------+--------------+
| 5ab8fe95-7ea3-4530-86de-facae9f52c71 | stack1     | CREATE_COMPLETE | 2016-12-07T06:31:07 | None         |
+--------------------------------------+------------+-----------------+---------------------+--------------+
[root@controller ~(keystone_admin)]#
[root@controller ~(keystone_admin)]#


See more information on the Stack which is created
===================================================
[root@controller ~(keystone_admin)]# heat stack-show stack1
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------+
| Property              | Value                                                                                                                            |
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------+
| capabilities          | []                                                                                                                               |
| creation_time         | 2016-12-07T06:31:07                                                                                                              |
| description           | Simple template to deploy a single compute instance                                                                              |
| disable_rollback      | True                                                                                                                             |
| id                    | 5ab8fe95-7ea3-4530-86de-facae9f52c71                                                                                             |
| links                 | http://controller.example.com:8004/v1/f84cfac47c10472fb36c56dc149d3caa/stacks/stack1/5ab8fe95-7ea3-4530-86de-facae9f52c71 (self) |
| notification_topics   | []                                                                                                                               |
| outputs               | [                                                                                                                                |
|                       |   {                                                                                                                              |
|                       |     "output_value": "stack1-my_instance-rrmsgn57cvxy",                                                                           |
|                       |     "output_key": "instance_name",                                                                                               |
|                       |     "description": "Name of the instance"                                                                                        |
|                       |   },                                                                                                                             |
|                       |   {                                                                                                                              |
|                       |     "output_value": "172.16.16.5",                                                                                               |
|                       |     "output_key": "instance_ip",                                                                                                 |
|                       |     "description": "IP address of the instance"                                                                                  |
|                       |   }                                                                                                                              |
|                       | ]                                                                                                                                |
| parameters            | {                                                                                                                                |
|                       |   "OS::project_id": "f84cfac47c10472fb36c56dc149d3caa",                                                                          |
|                       |   "OS::stack_id": "5ab8fe95-7ea3-4530-86de-facae9f52c71",                                                                        |
|                       |   "OS::stack_name": "stack1",                                                                                                    |
|                       |   "image": "Cirros",                                                                                                             |
|                       |   "key": "key1",                                                                                                                 |
|                       |   "private_network": "internal1",                                                                                                |
|                       |   "flavor": "m1.tiny"                                                                                                            |
|                       | }                                                                                                                                |
| parent                | None                                                                                                                             |
| stack_name            | stack1                                                                                                                           |
| stack_owner           | None                                                                                                                             |
| stack_status          | CREATE_COMPLETE                                                                                                                  |
| stack_status_reason   | Stack CREATE completed successfully                                                                                              |
| stack_user_project_id | f7e5c9e4b80f4f1faf411c95880c1309                                                                                                 |
| tags                  | null                                                                                                                             |
| template_description  | Simple template to deploy a single compute instance                                                                              |
| timeout_mins          | None                                                                                                                             |
| updated_time          | None                                                                                                                             |
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------+
[root@controller ~(keystone_admin)]#


Association of the floating IP to the instance
================================================
Asscoiate a floating IP to the instance. Create a floating IP if there are no free floating IPs available.

Get the information on all the networks, here the external is the external network in which the floating IP will get created.

[root@controller ~(keystone_admin)]# neutron net-list
+--------------------------------------+-----------+-------------------------------------------------------+
| id                                   | name      | subnets                                               |
+--------------------------------------+-----------+-------------------------------------------------------+
| 60ca0e7b-3c40-482c-92d1-3b4da4126e9d | testnet1  | 8fd08288-a83b-44ee-ac04-f8b88fc69628 172.16.15.0/24   |
| 8649a5a8-c137-444d-950d-e0c0569e3ee4 | internal1 | bd5b5a96-eaf5-45a8-93e5-6be837e61414 172.16.16.0/24   |
| 001ac29b-1f5f-4498-a44b-de0f809d2322 | external  | cf1da694-4a75-4768-b758-859dc750a02d 192.168.205.0/24 |
+--------------------------------------+-----------+-------------------------------------------------------+
[root@controller ~(keystone_admin)]# neutron floatingip-create externla^C
[root@controller ~(keystone_admin)]#


Create a floating IP in the external network
=============================================

[root@controller ~(keystone_admin)]# neutron floatingip-create external
Created a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| description         |                                      |
| fixed_ip_address    |                                      |
| floating_ip_address | 192.168.205.105                      |
| floating_network_id | 001ac29b-1f5f-4498-a44b-de0f809d2322 |
| id                  | ab40dcbd-1082-424a-b0fb-135b8756bc91 |
| port_id             |                                      |
| router_id           |                                      |
| status              | DOWN                                 |
| tenant_id           | f84cfac47c10472fb36c56dc149d3caa     |
+---------------------+--------------------------------------+
[root@controller ~(keystone_admin)]#


Ensure the floating IP is listed and that the UUID of floating IP appears as free floating IP
===============================================================

[root@controller ~(keystone_admin)]# neutron floatingip-list
+--------------------------------------+------------------+---------------------+--------------------------------------+
| id                                   | fixed_ip_address | floating_ip_address | port_id                              |
+--------------------------------------+------------------+---------------------+--------------------------------------+
| 816e8bf9-7dec-4011-9333-f761b8b95c15 | 172.16.15.3      | 192.168.205.104     | 69d9cde5-e06d-4ecf-9f87-0f91cb06a653 |
| ab40dcbd-1082-424a-b0fb-135b8756bc91 |                  | 192.168.205.105     |                                      |
| c1e1d2a7-7f5c-4b51-938e-eb064d5d1939 | 172.16.16.3      | 192.168.205.103     | 315b610f-54e1-404c-bac3-048feb37bc23 |
+--------------------------------------+------------------+---------------------+--------------------------------------+


See information of the new instance get the Private network IP of the same.
===============================================================

[root@controller ~(keystone_admin)]# nova list
+--------------------------------------+---------------------------------+--------+------------+-------------+----------------------------------------+
| ID                                   | Name                            | Status | Task State | Power State | Networks                               |
+--------------------------------------+---------------------------------+--------+------------+-------------+----------------------------------------+
| a2c265bb-242c-40ae-ae9c-98849e459bf7 | instance1                       | ACTIVE | -          | Running     | internal1=172.16.16.3, 192.168.205.103 |
| 5d767783-933e-423d-848f-37e816fa05b3 | stack1-my_instance-rrmsgn57cvxy | ACTIVE | -          | Running     | internal1=172.16.16.5                  |
+--------------------------------------+---------------------------------+--------+------------+-------------+----------------------------------------+

Find the neutron port for the instance this information is needed to associate the floating IP to the port of the instance
===============================================================
[root@controller ~(keystone_admin)]# neutron port-list | grep -i 172.16.16.5
| 42304108-f397-4de7-8936-5f011e4be5f3 |      | fa:16:3e:ff:52:0a | {"subnet_id": "bd5b5a96-eaf5-45a8-93e5-6be837e61414", "ip_address": "172.16.16.5"}     |
[root@controller ~(keystone_admin)]#
[root@controller ~(keystone_admin)]#


Associate the floating IP
=================
[root@controller ~(keystone_admin)]# neutron floatingip-associate ab40dcbd-1082-424a-b0fb-135b8756bc91 42304108-f397-4de7-8936-5f011e4be5f3
Associated floating IP ab40dcbd-1082-424a-b0fb-135b8756bc91
[root@controller ~(keystone_admin)]#

Ensure the floating IP is seen to be associated to the instance as in the nova list
========================================================================

[root@controller ~(keystone_admin)]# nova list
+--------------------------------------+---------------------------------+--------+------------+-------------+----------------------------------------+
| ID                                   | Name                            | Status | Task State | Power State | Networks                               |
+--------------------------------------+---------------------------------+--------+------------+-------------+----------------------------------------+
| a2c265bb-242c-40ae-ae9c-98849e459bf7 | instance1                       | ACTIVE | -          | Running     | internal1=172.16.16.3, 192.168.205.103 |
| 5d767783-933e-423d-848f-37e816fa05b3 | stack1-my_instance-rrmsgn57cvxy | ACTIVE | -          | Running     | internal1=172.16.16.5, 192.168.205.105 |
+--------------------------------------+---------------------------------+--------+------------+-------------+----------------------------------------+
[root@controller ~(keystone_admin)]#


you can now use the keys and SSH to the instance.

Please note that the security group for the instance has to have the ingress SSH allowed for a successful SSH to the instance

Friday, September 23, 2016

Neutron OpenvSwitch L2 agent on the nova Node not appearing in the neutron agent-list. MItaka huge time difference between the controller and the nova server



Neutron OpenvSwitch L2 agent on the nova Node not appearing in the neutron agent-list.


root@labnova:~[root@labnova ~]# neutron agent-list
+--------------------------------------+----------------------+------------------------+-------------------+-------+----------------+---------------------------+
| id                                   | agent_type           | host                   | availability_zone | alive | admin_state_up | binary                    |
+--------------------------------------+----------------------+------------------------+-------------------+-------+----------------+---------------------------+
| 31947b08-c7e4-4136-9559-f9d8bc31e1fb | DHCP agent           | labneutron.example.com | nova              | :-)   | True           | neutron-dhcp-agent        |
| 9d6bdf0b-3b11-4f46-9d67-ae49bc2ff968 | Loadbalancerv2 agent | labneutron.example.com |                   | :-)   | True           | neutron-lbaasv2-agent     |
| a94c66c6-d037-4080-ac0d-5cc8e8719729 | L3 agent             | labneutron.example.com | nova              | :-)   | True           | neutron-l3-agent          |
| c0c42fcd-5d6a-4618-8b3a-b71b79a639dc | Metadata agent       | labneutron.example.com |                   | :-)   | True           | neutron-metadata-agent    |
| c96830ca-7807-404c-8168-0627f1950afd | Open vSwitch agent   | labneutron.example.com |                   | :-)   | True           | neutron-openvswitch-agent |
| e49cea55-70cf-49b7-a8fe-91f2f227ebe8 | Open vSwitch agent   | labnova.example.com    |                   | xxx   | True           | neutron-openvswitch-agent |
+--------------------------------------+----------------------+------------------------+-------------------+-------+----------------+---------------------------+

Meanwhile appearing in the /var/log/neutron/server.log on the controller node.
2016-09-23 15:02:13.737 3828 ERROR neutron.db.agents_db [req-9628b672-43b1-4883-89a3-fec7511cb854 - - - - -] Message received from the host: labnova.example.com during the registration of Open vSwitch agent has a timestamp: 2016-09-20T22:55:24.425782. This differs from the current server timestamp: 2016-09-23T19:02:13.736852 by 245209.31107 seconds, which is more than the threshold agent downtime: 75.
2016-09-23 15:02:20.677 3828 WARNING neutron.plugins.ml2.drivers.l2pop.mech_driver [req-9628b672-43b1-4883-89a3-fec7511cb854 - - - - -] Unable to retrieve active L2 agent on host labnova.example.com
2016-09-23 15:02:21.138 3828 WARNING neutron.plugins.ml2.drivers.l2pop.mech_driver [req-9628b672-43b1-4883-89a3-fec7511cb854 - - - - -] Unable to retrieve active L2 agent on host labnova.example.com
2016-09-23 15:02:21.633 3828 WARNING neutron.plugins.ml2.drivers.l2pop.mech_driver [req-9628b672-43b1-4883-89a3-fec7511cb854 - - - - -] Unable to retrieve active L2 agent on host labnova.example.com


On the nova server the service neutron-openvswitch-agent had already been running.

#systemctl –a | grep -i  neutron-openvswitch-agent.service                                                                              loaded    active   running  

There was a date difference of 3 days on the controller and the nova node.
root@labnova:/etc/neutron/plugins/ml2[root@labnova ml2]# date
Tue Sep 20 18:57:26 EDT 2016

At the same time the date was on the controller server as

root@labcontroller:/var/log/neutron[root@labcontroller neutron]# date
Fri Sep 23 15:13:39 EDT 2016


The fix:

The fix was to restart the chronyd on the nova server to sync the time correctly with the NTP Source and restarting the neutron-openvswitch-agent.service running on the nova server labnova.

Sunday, July 10, 2016

Bad configuration issue + OpenStack Mitaka on Cinder service Causing "cinder service-list ERROR: Service Unavailable (HTTP 503)" Authorization failed. The request you have made requires authentication. from 192.168.100.138

Bad configuration issue + OpenStack Mitaka on Cinder service Causing "cinder service-list ERROR: Service Unavailable (HTTP 503)" Authorization failed. The request you have made requires authentication. from 192.168.100.138

Error on the command prompt as returned

ERROR: Service Unavailable (HTTP 503)"

Error on the controller server /var/log/keystone/keystone.log


2016-07-10 17:44:35.346 3533 INFO keystone.common.wsgi [req-220e3bf4-2382-4f2c-8c46-55539f9ae098 - - - - -] POST http://controller1.example.com:35357/v3/auth/tokens
2016-07-10 17:44:35.998 3533 WARNING keystone.common.wsgi [req-220e3bf4-2382-4f2c-8c46-55539f9ae098 - - - - -] Authorization failed. The request you have made requires authentication. from 192.168.100.138

The fix was as below

This was a wrong configuration in which the /etc/cinder/cinder.conf on the Controller server controller1.example.com and /etc/nova/nova.conf had been correct as per the Mitaka installation document on CentOS (www.openstack.org). Also on the cinder server bstore1.example.com the file /etc/cinder/cinder.conf had been correctly set. 

The Database user cinder and the OpenStack service user cinder also had been with the passwords and the same had been being used in the configuration files /etc/cinder/cinder.conf.

Finally this issue was found out be accounted by the fact that after the cinder user was created on the controller server, the same was not given an admin role to the project service.

This was fixed by providing the admin role to the cinder user on the project service on the controller server and this had made things run fine.

# . admin-openrc -> to source the admin credentials.

# openstack role add --project service --user cinder admin

After this things started working as 

[root@controller1 ~]# cinder service-list
+------------------+-------------------------+------+---------+-------+----------------------------+-----------------+
|      Binary      |           Host          | Zone |  Status | State |         Updated_at         | Disabled Reason |
+------------------+-------------------------+------+---------+-------+----------------------------+-----------------+
| cinder-scheduler | controller1.example.com | nova | enabled |   up  | 2016-07-10T22:05:35.000000 |        -        |
+------------------+-------------------------+------+---------+-------+----------------------------+-----------------+
[root@controller1 ~]#
[root@controller1 ~]#
[root@controller1 ~]#
[root@controller1 ~]#
[root@controller1 ~]#
[root@controller1 ~]#
[root@controller1 ~]# cinder list
+----+--------+------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+----+--------+------+------+-------------+----------+-------------+
+----+--------+------+------+-------------+----------+-------------+

[root@controller1 ~]#



Also the /var/log/keystone/keystone.log has started showing the correct authorization of the cinder 

2016-07-10 17:47:28.388 3536 INFO keystone.token.providers.fernet.utils [req-2ceda5e9-1ca9-41d9-b0c9-40bf8e60208d - - - - -] Loaded 2 encryption keys (max_active_keys=3) from: /etc/keystone/fernet-keys/
2016-07-10 17:47:28.422 3537 INFO keystone.token.providers.fernet.utils [req-a2e5f7c5-6c34-4202-8b4d-2344541b5ea2 - - - - -] Loaded 2 encryption keys (max_active_keys=3) from: /etc/keystone/fernet-keys/
2016-07-10 17:47:28.668 3537 INFO keystone.common.wsgi [req-a2e5f7c5-6c34-4202-8b4d-2344541b5ea2 31a638cfa16a42019ef1d7255a157a47 95b5e18a71274873b72d7faba0cb4365 - ca7e0958d3c84cc8800c8227fe84823a ca7e0958d3c84cc8800c8227fe84823a] GET http://controller1.example.com:35357/v3/auth/tokens
2016-07-10 17:47:28.670 3537 INFO keystone.token.providers.fernet.utils [req-a2e5f7c5-6c34-4202-8b4d-2344541b5ea2 31a638cfa16a42019ef1d7255a157a47 95b5e18a71274873b72d7faba0cb4365 - ca7e0958d3c84cc8800c8227fe84823a ca7e0958d3c84cc8800c8227fe84823a] Loaded 2 encryption keys (max_active_keys=3) from: /etc/keystone/fernet-keys/
2016-07-10 17:47:28.908 3537 INFO keystone.token.providers.fernet.utils [req-a2e5f7c5-6c34-4202-8b4d-2344541b5ea2 31a638cfa16a42019ef1d7255a157a47 95b5e18a71274873b72d7faba0cb4365 - ca7e0958d3c84cc8800c8227fe84823a ca7e0958d3c84cc8800c8227fe84823a] Loaded 2 encryption keys (max_active_keys=3) from: /etc/keystone/fernet-keys/


Note: Similar issue can be caused when using passwords for the cinder user being different in the cinder database and in cinder user in the service creation to those being used in the cinder configuration file /etc/cinder/cinder.conf on the controller also the cinder servers.



Wednesday, July 6, 2016

Issues starting the nova conductor service OpenStack Mitaka: AMQP server controller1.example.com:5672 closed the connection. Check login credentials: Socket closed




Issues starting the nova conductor service on the controller node. This results as a part of the misconfiguration of the nova conductor in /etc/nova/nova.conf on the controller node of OpenStack.

As usual the best place to see the reasons for problems with the OpenStack Nova services on the controller is at the location /var/log/nova/*. See the logs there.


The below message appears in /var/log/nova/nova-conductor.log

==> nova-conductor.log <==
2016-07-06 06:32:20.891 27161 ERROR oslo.messaging._drivers.impl_rabbit [req-7bff235c-2535-4d18-b9e0-351c6e23557c - - - - -] AMQP server controller1.example.com:5672 closed the connection. Check login credentials: Socket closed

This means that the user mentioned on the Nova Configuration user as the user to be used to access the Rabbitmq either is not present or if present has a password which does not match with the rabbitmq configuration.


My Nova Configuration file looked like this (/etc/nova/nova.conf on the controller server)

[DEFAULT]
rpc_backend = rabbit

[oslo_messaging_rabbit]
rabbit_host = controller1.example.com
rabbit_userid = openstack
rabbit_password = XXXXXXX



The Fix:



1) In case the user for rabbitmq access as mentioned here as "openstack" is not created then create the user on the controller server in the setup as 

rabbitmqctl add_user openstack XXXXXXX

where XXXXXXX is a password which you want to give to the user

2) In case there is a password mismatch between the one given in /etc/nova/nova.conf and the actual rabbitmq password in the rabbitmq password you can set a new password for the user (openstack here) and update the same in the NOVA configuration file /etc/nova/nova.conf

rabbitmqctl change_passwd <user> <password>

Refer to the man pages of rabbitmqctl for more.

root@controller1 nova]# rabbitmqctl --help | grep -i use
Error: could not recognise command
the correct suffix to use after the "@" sign. See rabbitmq-server(1) for
    add_user <username> <password>
    delete_user <username>
    change_password <username> <newpassword>
    clear_password <username>
            authenticate_user <username> <password>
    set_user_tags <username> <tag> ...
    list_users
    set_permissions [-p <vhost>] <user> <conf> <write> <read>
    clear_permissions [-p <vhost>] <username>
    list_user_permissions <username>
channels, protocol, auth_mechanism, user, vhost, timeout, frame_max,
user, vhost, transactional, confirm, consumer_count, messages_unacknowledged,
[root@controller1 nova]#
[root@controller1 nova]#
[root@controller1 nova]# rabbitmqctl change_passwd nova XXXXXXX

OpenStack Mitaka Issue starting the Nova API service on the controller "ERROR nova MissingRequiredOptions: Auth plugin requires parameters which were not given: auth_url"



OpenStack Mitaka Issue starting the Nova API service on the controller "ERROR nova MissingRequiredOptions: Auth plugin requires parameters which were not given: auth_url"


This issue was faced at the time of starting the nova-api service on the controller node on the OpenStack Setup. The reason was a misconfigured nova api.

As usual for nova related service best place to look out are the logs in /var/log/nova/* on the controller machine.


The below messages appear in the /var/log/nova/nova-api log


==> nova-api.log <==

2016-07-06 06:32:16.947 27284 WARNING oslo_reports.guru_meditation_report [-] Guru mediation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
2016-07-06 06:32:17.505 27284 INFO oslo_service.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2016-07-06 06:32:18.244 27284 INFO nova.api.openstack [-] Loaded extensions: ['extensions', 'flavors', 'image-metadata', 'image-size', 'images', 'ips', 'limits', 'os-access-ips', 'os-admin-actions', 'os-admin-password', 'os-agents', 'os-aggregates', 'os-assisted-volume-snapshots', 'os-attach-interfaces', 'os-availability-zone', 'os-baremetal-nodes', 'os-block-device-mapping', 'os-cells', 'os-certificates', 'os-cloudpipe', 'os-config-drive', 'os-console-auth-tokens', 'os-console-output', 'os-consoles', 'os-create-backup', 'os-deferred-delete', 'os-disk-config', 'os-evacuate', 'os-extended-availability-zone', 'os-extended-server-attributes', 'os-extended-status', 'os-extended-volumes', 'os-fixed-ips', 'os-flavor-access', 'os-flavor-extra-specs', 'os-flavor-manage', 'os-flavor-rxtx', 'os-floating-ip-dns', 'os-floating-ip-pools', 'os-floating-ips', 'os-floating-ips-bulk', 'os-fping', 'os-hide-server-addresses', 'os-hosts', 'os-hypervisors', 'os-instance-actions', 'os-instance-usage-audit-log', 'os-keypairs', 'os-lock-server', 'os-migrate-server', 'os-migrations', 'os-multinic', 'os-multiple-create', 'os-networks', 'os-networks-associate', 'os-pause-server', 'os-personality', 'os-preserve-ephemeral-rebuild', 'os-quota-class-sets', 'os-quota-sets', 'os-remote-consoles', 'os-rescue', 'os-scheduler-hints', 'os-security-group-default-rules', 'os-security-groups', 'os-server-diagnostics', 'os-server-external-events', 'os-server-groups', 'os-server-password', 'os-server-usage', 'os-services', 'os-shelve', 'os-simple-tenant-usage', 'os-suspend-server', 'os-tenant-networks', 'os-used-limits', 'os-user-data', 'os-virtual-interfaces', 'os-volumes', 'server-metadata', 'server-migrations', 'servers', 'versions']
2016-07-06 06:32:18.257 27284 CRITICAL nova [-] MissingRequiredOptions: Auth plugin requires parameters which were not given: auth_url
2016-07-06 06:32:18.257 27284 ERROR nova Traceback (most recent call last):
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/bin/nova-api", line 10, in <module>
2016-07-06 06:32:18.257 27284 ERROR nova     sys.exit(main())
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/nova/cmd/api.py", line 57, in main
2016-07-06 06:32:18.257 27284 ERROR nova     server = service.WSGIService(api, use_ssl=should_use_ssl)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/nova/service.py", line 366, in __init__
2016-07-06 06:32:18.257 27284 ERROR nova     self.app = self.loader.load_app(name)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/nova/wsgi.py", line 497, in load_app
2016-07-06 06:32:18.257 27284 ERROR nova     return deploy.loadapp("config:%s" % self.config_path, name=name)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2016-07-06 06:32:18.257 27284 ERROR nova     return loadobj(APP, uri, name=name, **kw)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2016-07-06 06:32:18.257 27284 ERROR nova     return context.create()
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-07-06 06:32:18.257 27284 ERROR nova     return self.object_type.invoke(self)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2016-07-06 06:32:18.257 27284 ERROR nova     **context.local_conf)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-07-06 06:32:18.257 27284 ERROR nova     val = callable(*args, **kw)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/nova/api/openstack/urlmap.py", line 160, in urlmap_factory
2016-07-06 06:32:18.257 27284 ERROR nova     app = loader.get_app(app_name, global_conf=global_conf)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2016-07-06 06:32:18.257 27284 ERROR nova     name=name, global_conf=global_conf).create()
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-07-06 06:32:18.257 27284 ERROR nova     return self.object_type.invoke(self)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2016-07-06 06:32:18.257 27284 ERROR nova     **context.local_conf)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-07-06 06:32:18.257 27284 ERROR nova     val = callable(*args, **kw)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/nova/api/auth.py", line 79, in pipeline_factory_v21
2016-07-06 06:32:18.257 27284 ERROR nova     return _load_pipeline(loader, local_conf[CONF.auth_strategy].split())
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/nova/api/auth.py", line 62, in _load_pipeline
2016-07-06 06:32:18.257 27284 ERROR nova     app = filter(app)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 1100, in auth_filter
2016-07-06 06:32:18.257 27284 ERROR nova     return AuthProtocol(app, conf)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 682, in __init__
2016-07-06 06:32:18.257 27284 ERROR nova     self._identity_server = self._create_identity_server()
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 1050, in _create_identity_server
2016-07-06 06:32:18.257 27284 ERROR nova     auth_plugin = self._get_auth_plugin()
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 995, in _get_auth_plugin
2016-07-06 06:32:18.257 27284 ERROR nova     return plugin_loader.load_from_options_getter(getter)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/keystoneauth1/loading/base.py", line 148, in load_from_options_getter
2016-07-06 06:32:18.257 27284 ERROR nova     return self.load_from_options(**kwargs)
2016-07-06 06:32:18.257 27284 ERROR nova   File "/usr/lib/python2.7/site-packages/keystoneauth1/loading/base.py", line 123, in load_from_options
2016-07-06 06:32:18.257 27284 ERROR nova     raise exceptions.MissingRequiredOptions(missing_required)
2016-07-06 06:32:18.257 27284 ERROR nova MissingRequiredOptions: Auth plugin requires parameters which were not given: auth_url
2016-07-06 06:32:18.257 27284 ERROR nova

The fix:


This comes if you have not define or improperly defined the auth_url in the /etc/nova/nova.conf

in my case this has been in /etc/nova/nova.conf  as

auth_url = http://controller1.example.com:35357

Also is worth noting another entry in the file as auth_uri in the /etc/nova/nova.conf 

as

auth_uri = http://controller1.example.com:5000