Compute

Akraino :: Tempest :: Compute

Akraino :: Tempest :: Compute :: Setup

1. Identify Image, Flavor and Network

[wrsroot@controller-0 ~(keystone_admin)]$ openstack image list
+--------------------------------------+---------------+--------+
| ID                                   | Name          | Status |
+--------------------------------------+---------------+--------+
| 3c8995af-4223-42b6-8872-7ed75bf19ddd | cirros        | active |
| 3b37f2f6-a923-4025-945d-4df5a1a5526b | cirrosx       | active |
| 852f6a6b-1be1-4598-8769-36538bbfd9d7 | debian        | active |
| 59cd540a-78eb-40d3-9a8c-b12a5e46ac74 | ubuntudesktop | active |
+--------------------------------------+---------------+--------+

[wrsroot@controller-0 ~(keystone_admin)]$ openstack flavor list
+----+---------------+-------+------+-----------+-------+-----------+
| ID | Name          |   RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+---------------+-------+------+-----------+-------+-----------+
| 1  | flavor.nano   |    64 |    1 |         0 |     1 | True      |
| 2  | flavor.micro  |   128 |    2 |         0 |     1 | True      |
| 3  | flavor.tiny   |   512 |    4 |         0 |     1 | True      |
| 4  | flavor.small  |  2048 |   20 |         0 |     1 | True      |
| 5  | flavor.medium |  4096 |   40 |         0 |     2 | True      |
| 6  | flavor.large  |  8192 |   80 |         0 |     4 | True      |
| 7  | flavor.xlarge | 16384 |  160 |         0 |     8 | True      |
| 8  | flavor.limit  |  5120 |    4 |         0 |     2 | True      |
| 9  | flavor.custom |  2048 |    4 |         0 |     2 | True      |
+----+---------------+-------+------+-----------+-------+-----------+

[wrsroot@controller-0 ~(keystone_admin)]$ openstack network list
+--------------------------------------+----------------+--------------------------------------+
| ID                                   | Name           | Subnets                              |
+--------------------------------------+----------------+--------------------------------------+
| 012afdaa-92f1-401f-8a54-b3e15d43f82f | network.nano   | ec0e2d54-0c70-4a6d-bdcb-1dc7b224ff71 |
| 37ff3a19-cabd-4f1b-a9a3-82c85422746d | network.micro  | 4a91686f-9a37-49d6-8696-40e00b633d48 |
| 54241f97-de23-4e97-8f76-be35e614e0c1 | network.small  | 08bd4390-fdfc-414f-a1e8-a92a3ad753b8 |
| b3d3d537-1756-4f64-b608-691f5113079c | network.xlarge | a05d3dc0-327a-4e6c-9e18-29bc60a63ad8 |
| bb03d10c-d802-46f2-95df-880b778241e0 | network.tiny   | b8eec7c4-e762-460d-8705-427fdf4c31e0 |
| c634b744-36c5-4e63-8a72-3385a075595a | network.large  | 18165934-7f48-4fa5-93f4-c7a945c315c4 |
| d1d2a65d-3164-4691-97e5-305ab7bac657 | network.medium | 54cb96d2-2d1f-4f4c-9fba-bdd449b76534 |
| d20ab868-0740-4910-a094-b1215990e018 | network.limit  | d694a65b-d827-4695-8958-3aa966de2847 |
| f924fd06-acc1-4344-91ba-b6b6ff06c9c6 | network.custom | 7bb100ed-e251-4eef-83ab-30acd38aaf70 |
+--------------------------------------+----------------+--------------------------------------+

1. Modify etc/tempest.conf under compute section to reflect these changes:

[compute]

image_ref = 3c8995af-4223-42b6-8872-7ed75bf19ddd
image_ref_alt = 3b37f2f6-a923-4025-945d-4df5a1a5526b
flavor_ref = 1
flavor_ref_alt = 2
fixed_network_name = network.nano
endpoint_type = publicURL

The following are referenced in other tempest.conf files:

[compute]
...
allow_tenant_isolation = true
allow_tenant_reuse = true
max_microversion = latest
build_timeout = 196

1. Modify etc/tempest.conf under network section:

[network]

#
# From tempest.config
#

# Catalog type of the Neutron service. (string value)
catalog_type = network

# The network region name to use. If empty, the value of
# identity.region is used instead. If no such region is found in the
# service catalog, the first found one is used. (string value)
#region =

# The endpoint type to use for the network service. (string value)
# Possible values:
# public - <No description provided>
# admin - <No description provided>
# internal - <No description provided>
# publicURL - <No description provided>
# adminURL - <No description provided>
# internalURL - <No description provided>
endpoint_type =    admin 

# The cidr block to allocate project ipv4 subnets from (string value)
#project_network_cidr = 10.100.0.0/16

# The mask bits for project ipv4 subnets (integer value)
#project_network_mask_bits = 28

# The cidr block to allocate project ipv6 subnets from (string value)
#project_network_v6_cidr = 2003::/48

# The mask bits for project ipv6 subnets (integer value)
#project_network_v6_mask_bits = 64

# Whether project networks can be reached directly from the test
# client. This must be set to True when the 'fixed' connect_method is
# selected. (boolean value)
project_networks_reachable = true

# Id of the public network that provides external connectivity (string
# value)
public_network_id = 012afdaa-92f1-401f-8a54-b3e15d43f82f

# Default floating network name. Used to allocate floating IPs when
# neutron is enabled. (string value)
#floating_network_name = <None>

# Id of the public router that provides external connectivity. This
# should only be used when Neutron's 'allow_overlapping_ips' is set to
# 'False' in neutron.conf. usually not needed past 'Grizzly' release
# (string value)
#public_router_id =

# Timeout in seconds to wait for network operation to complete.
# (integer value)
#build_timeout = 300

# Time in seconds between network operation status checks. (integer
# value)
#build_interval = 1

# List of dns servers which should be used for subnet creation (list
# value)
#dns_servers = 8.8.8.8,8.8.4.4

# vnic_type to use when launching instances with pre-configured ports.
# Supported ports are: ['normal','direct','macvtap'] (string value)
# Possible values:
# <None> - <No description provided>
# normal - <No description provided>
# direct - <No description provided>
# macvtap - <No description provided>
#port_vnic_type = <None>

# port profile to use when launching instances with pre-configured
# ports. (dict value)
#port_profile =

# List of ip pools for subnetpools creation (list value)
#default_network = 1.0.0.0/16,2.0.0.0/16

# The environment does not support network separation between tenants.
# (boolean value)
#shared_physical_network = false

1. Modify etc/tempest.conf under service_available section:

[service_available]
...
neutron = true

Akraino :: Tempest :: Compute :: Number of Tests

Check the number of tests in both all and smoke types

user@workstation:~/tempest$ tempest run --smoke --list-tests  | grep compute | wc -l
21
user@workstation:~/tempest$ tempest run --list-tests  | grep compute | wc -l
620

Akraino :: Tempest :: Compute :: Smoke

Run the smoke type tests

user@workstation:~/tempest$ tempest run --smoke --list-tests | grep compute > testslist

user@workstation:~/tempest$ cat testslist 
tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_add_remove_fixed_ip[id-c7e0e60b-ee45-43d0-abeb-8596fd42a2f9,network,smoke]
tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_verify_server_details[id-5de47127-9977-400a-936f-abcfbec1218f,smoke]
tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_list_servers[id-9a438d88-10c6-4bcd-8b5b-5b6e25e1346f,smoke]
tempest.api.compute.test_versions.TestVersions.test_get_version_details[id-b953a29e-929c-4a8e-81be-ec3a7e03cb76,smoke]
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard[id-2cb1baf6-ac8d-4429-bf0d-ba8a0ba53e32,smoke]
tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance[compute,id-fff42874-7db5-4487-a8e1-ddda5fb5288d,smoke]
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_list[id-a6154130-5a55-4850-8be4-5e9e796dbf17,smoke]
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_list_servers[id-9a438d88-10c6-4bcd-8b5b-5b6e25e1346f,smoke]
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses_by_network[id-87bbc374-5538-4f64-b673-2b0e4443cc30,network,smoke]
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_verify_server_details[id-5de47127-9977-400a-936f-abcfbec1218f,smoke]
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke]
tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes[compute,id-9cecbe35-b9d4-48da-a37e-7ce70aa43d30,network,smoke]
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_verify_server_details[id-5de47127-9977-400a-936f-abcfbec1218f,smoke]
tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON.test_security_groups_create_list_delete[id-eb2b087d-633d-4d0d-a7bd-9e6ba35b32de,smoke]
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_list_flavors[id-e36c0eaa-dff5-4082-ad1f-3f9a80aa3f59,smoke]
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses[id-6eb718c0-02d9-4d5e-acd1-4e0c269cef39,network,smoke]
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_get_flavor[id-1f12046b-753d-40d2-abb6-d8eb8b30cb2f,smoke]
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_list_servers[id-9a438d88-10c6-4bcd-8b5b-5b6e25e1346f,smoke]
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_create[id-850795d7-d4d3-4e55-b527-a774c0123d3a,smoke]
tempest.api.compute.test_versions.TestVersions.test_list_api_versions[id-6c0a0990-43b6-4529-9b61-5fd8daf7c55c,smoke]
user@workstation:~/tempest$

user@workstation:~/tempest$ tempest run --concurrency 1 --load-list testslist
...
...
======
Totals
======
Ran: 20 tests in 170.0000 sec.
 - Passed: 14
 - Skipped: 3
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 3
Sum of execute time for each test: 44.3492 sec.

==============
Worker Balance
==============
 - Worker 0 (20 tests) => 0:02:46.042786

Passed

tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_verify_server_details
tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_list_servers
tempest.api.compute.test_versions.TestVersions.test_get_version_details
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_list
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses_by_network
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_verify_server_details
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_verify_server_details
tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON.test_security_groups_create_list_delete
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_list_flavors
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_get_flavor
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_list_servers
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_create
tempest.api.compute.test_versions.TestVersions.test_list_api_versions

Failed

tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_add_remove_fixed_ip
tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_list_servers
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops
tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops]

Akraino :: Tempest :: Compute :: Smoke :: Failed

{0} setUpClass (tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume) [0.000000s] ... FAILED
    tempest.lib.exceptions.EndpointNotFound: Endpoint not found
    Details: No matching service found in the catalog.

[wrsroot@controller-0 ~(keystone_admin)]$ openstack endpoint list
| ID                               | Region    | Service Name | Service Type   | Enabled | Interface | URL                                          |
+----------------------------------+-----------+--------------+----------------+---------+-----------+----------------------------------------------+
| cd0a6e0c4730435da86dbf46e2cb8892 | RegionOne | glance       | image          | True    | admin     | http://192.168.204.2:9292                    |
| 7a0418c205114e07bfec6c43b0e2ed21 | RegionOne | glance       | image          | True    | internal  | http://192.168.204.2:9292                    |
| e1d34ba0b15f48b0b404308dc8c27b4d | RegionOne | glance       | image          | True    | public    | http://10.10.10.2:9292                       |

Enabled Cinder Service 

All [service_available] enabled
All endpoint_type enabled
All catalog_type enabled
All region enabled

        """Base URL from catalog

        If scope is not 'project', it may be that there is not catalog in
        the auth_data. In such case, as long as the requested service is
        'identity', we can use the original auth URL to build the base_url.

        :param filters: Used to filter results

        Filters can be:

        - service: service type name such as compute, image, etc.
        - region: service region name
        - name: service name, only if service exists
        - endpoint_type: type of endpoint such as
            adminURL, publicURL, internalURL
        - api_version: the version of api used to replace catalog version
        - skip_path: skips the suffix path of the url and uses base URL

{0} setUpClass (tempest.api.volume.test_volumes_actions.VolumesActionsTest) [0.000000s] ... FAILED
    tempest.lib.exceptions.EndpointNotFound: Endpoint not found
    Details: No matching service found in the catalog.

{0} tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops [30.745910s] ... FAILED
    tempest.exceptions.BuildErrorException: Server 30e92ff0-9391-48b4-9732-949e8881a033 failed to build and is in ERROR status
    Details: {u'message': u'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 30e92ff0-9391-48b4-9732-949e8881a033. Last exception: Network requires port_security_enabled and subnet associated in order to apply security groups.', u'code': 500, u'details': u'  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 583, in build_instances\n    filter_properties, instances[0].uuid)\n  File "/usr/lib/python2.7/site-packages/nova/scheduler/utils.py", line 515, in populate_retry\n    raise exception.MaxRetriesExceeded(reason=msg)\n', u'created': u'2018-04-02T23:30:55Z'}

By looking into more details from the console log    

    2018-04-02 19:50:38,083 18699 INFO     [tempest.lib.common.fixed_network] (TestServerBasicOps:test_server_basic_ops) Found network {u'provider:physical_network': u'providernet-a', u'updated_at': u'2018-03-28T02:47:48Z', u'revision_number': 2, u'provider:network_type': u'vlan', u'id': u'012afdaa-92f1-401f-8a54-b3e15d43f82f', u'router:external': False, u'availability_zone_hints': [], u'availability_zones': [u'nova'], u'ipv4_address_scope': None, u'shared': False, u'project_id': u'ce3514d15bc64ed69b45552423350a4d', u'status': u'ACTIVE', u'subnets': [u'ec0e2d54-0c70-4a6d-bdcb-1dc7b224ff71'], u'description': u'', u'tags': [], u'ipv6_address_scope': None, u'provider:segmentation_id': 100, u'name': u'network.nano', u'admin_state_up': True, u'tenant_id': u'ce3514d15bc64ed69b45552423350a4d', u'created_at': u'2018-03-28T02:47:11Z', u'mtu': 1500, u'vlan_transparent': False} available for tenant

By disabling fixed_network_name we now receive:

# Name of the fixed network that is visible to all test projects. If
# multiple networks are available for a project, this is the network
# which will be used for creating servers if tempest does not create a
# network or a network is not specified elsewhere. It may be used for
# ssh validation only if floating IPs are disabled. (string value)
#fixed_network_name = network.nano

{0} tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops [14.611729s] ... ok     

2018-04-02 19:58:09.494 18847 INFO tempest.lib.common.dynamic_creds [-] Acquired dynamic creds:
 credentials: Credentials: {'username': u'tempest-TestServerBasicOps-943561569', 'project_name': u'tempest-TestServerBasicOps-943561569', 'project_domain_id': u'default', 'user_domain_id': u'default', 'tenant_id': u'efb9e61315a04a27a8de30410d767593', 'user_domain_name': u'Default', 'domain_name': u'Default', 'tenant_name': u'tempest-TestServerBasicOps-943561569', 'user_id': u'407e9a3ddf6d42ba97a0c401f5f31d30', 'project_id': u'efb9e61315a04a27a8de30410d767593', 'domain_id': u'default', 'project_domain_name': u'Default'}, Network: None, Subnet: None, Router: None

Components

• Compute

• Identity

• Image

• Network

• Object-Storage

• Volume

Akraino :: Tempest :: Compute :: Smoke :: Skipped

First and second, third runs... debug mode!

{0} setUpClass (tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON) ... SKIPPED: Neutron is required
{0} tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_add_remove_fixed_ip [17.257858s] ... ok

Passed after a second run :)

{0} setUpClass (tempest.scenario.test_network_basic_ops.TestNetworkBasicOps) ... SKIPPED: Neutron not available
{0} setUpClass (tempest.scenario.test_network_basic_ops.TestNetworkBasicOps) ... SKIPPED: Either project_networks_reachable must be "true", or public_network_id must be defined.

We look at the following options under etc/tempest.conf [network] section

# Whether project networks can be reached directly from the test
# client. This must be set to True when the 'fixed' connect_method is
# selected. (boolean value)
project_networks_reachable = true

# Id of the public network that provides external connectivity (string
# value)
#public_network_id = 

By enabling project_networks_reachable we now receive:

    tempest.lib.exceptions.UnexpectedResponseCode: Unexpected response code received
    Details: 503

Keep Testing!

{0} setUpClass (tempest.scenario.test_server_multinode.TestServerMultinode) ... SKIPPED: Less than 2 compute nodes, skipping multinode tests.
{0} setUpClass (tempest.scenario.test_server_multinode.TestServerMultinode) ... SKIPPED: Less than 2 compute nodes, skipping multinode tests.

We look at the following option under etc/tempest.conf [compute] section

# The minimum number of compute nodes expected. This will be utilized
# by some multinode specific tests to ensure that requests match the
# expected size of the cluster you are testing with. (integer value)
min_compute_nodes = X

By enabling to 5 we receive now:

        tempest.exceptions.BuildErrorException: Server f7d1d33a-7394-42d8-ba9e-e0eef2d062f9 failed to build and is in ERROR status
    Details: {u'message': u'[Errno 12] Cannot allocate memory', u'code': 500, u'details': u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2012, in _do_build_and_run_instance\n    filter_properties)\n  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2259, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'created': u'2018-04-03T00:21:20Z'}

By enabling to 2,3 we receive now:
 {0} tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes [46.846218s] ... ok

ToDo. Boundary tests using the maximum number of compute nodes available in the cluster.

Akraino :: Tempest :: Compute :: All

user@workstation:~/tempest$ tempest run --list-tests | grep compute > testslist

ToDo. Run all tests

======
Totals
======
Ran: 551 tests in 6390.0000 sec.
 - Passed: 354
 - Skipped: 74
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 123
Sum of execute time for each test: 5000.3678 sec.

==============
Worker Balance
==============
 - Worker 0 (551 tests) => 1:46:22.765223