# Tempest

## Tempest

> *This is a set of integration tests to be run against a live OpenStack cluster. Tempest has batteries of tests for OpenStack API validation, Scenarios, and other specific tests useful in validating an OpenStack deployment.* [Homepage](https://docs.openstack.org/tempest/latest/)

* [Testing your OpenStack with Tempest – Part 1](https://aptira.com/testing-openstack-tempest-part-1/)
* [Introduction to tempest](https://es.slideshare.net/openstackindia/introduction-to-tempest?next_slideshow=1)
* [Tempest](https://es.slideshare.net/ttakekawa/tempest-22843588)

> Oslo. To produce a set of python libraries containing code shared by OpenStack projects. The APIs provided by these libraries should be high quality, stable, consistent, documented and generally applicable. [Homepage](https://wiki.openstack.org/wiki/Oslo)

Type of Tests

* Smoke
* Positive
* Negative
* stress
* whiteBox

Components

* API
  * Compute
    * Admin
    * Certificates
    * Flavors
    * Floating IPs
    * Images
    * Keypairs
    * Limits
    * Security Groups
    * Servers
    * Test Extensions
    * Test Networks
    * Test Quotas
    * Test Versions
    * Volume
  * Identity
    * Admin
    * v2
  * Image
    * v1
    * v2
  * Network
    * Admin
    * Test Allow Address Pair
    * Test DHCP IPv6
    * Test Extensions
    * Test Extra DHCP Options
    * Test Floating IPs
    * Test Networks
    * Test Ports
    * Test Routers
    * Test routers Negative
    * Test Security Groups
    * Test Security Groups Negative
    * Test Service Providers
    * Test Tags
    * Test Versions
  * Object-Storage
    * Test Account Bulk
    * Test Account Quotas
    * Test Account Services
    * Test Account Services Negative
    * Test Container ACL
    * Test Container ACL Negative
    * Test Quotas
    * Test Container Services
    * Test Container StaticWeb
    * Test Container Sync
    * Test Cross Domain
    * Test Health Check 
    * Test Object Expiry
    * Test Object Formpost
    * Test Object Formpost Negative
    * Test Object Services
    * Test Object SLO
    * Test Object Temp URL
    * Test Object Version
  * Volume
    * Admin
    * Test Availability Zone
    * Test Extension
    * Test Image Metadata
    * Test Snapshot Metadata
    * Test Versions
    * Test Volume Absolute Limits
    * Test Volume Delete Cascade
    * Test Volume Metadata
    * Test Volume Transfer
    * Test Volumes Actions
    * Test Volumes Backup
    * Test Volumes Clone
    * Test Volumes Extend
    * Test Volumes Get
    * Test Volumes List
    * Test Volumes Negative
    * Test Volumes Snapshots
* Scenario
  * test\_aggregates\_basic\_ops
  * test\_encrypted\_cinder\_volumes
  * test\_minimum\_basic
  * test\_network\_advanced\_server\_ops
  * test\_network\_advanced\_server\_ops
  * test\_network\_v6
  * test\_object\_storage\_basic\_ops
  * test\_security\_groups\_basic\_ops
  * test\_server\_advanced\_ops
  * test\_server\_multinode
  * test\_shelve\_instance
  * test\_snapshot\_pattern
  * test\_stamp\_pattern
  * test\_volume\_boot\_pattern
  * test\_volume\_migrate\_attached

## Tempest :: Directory Structure

Tbd

## Tempest :: To Enable

* Nova cert
* Floating IP pool not found
* The microversion range\[2.32 - 2.32] of this test is out of the configuration range

## Tempest :: Installation

* [Quickstart](https://docs.openstack.org/tempest/latest/overview.html)

```bash
user@workstation:~$ git clone https://git.openstack.org/openstack/tempest
Cloning into 'tempest'...
remote: Counting objects: 96572, done.
remote: Compressing objects: 100% (53534/53534), done.
remote: Total 96572 (delta 74431), reused 63135 (delta 42741)
Receiving objects: 100% (96572/96572), 13.70 MiB | 260.00 KiB/s, done.
Resolving deltas: 100% (74431/74431), done.
Checking connectivity... done.
user@workstation:~$
```

```bash
user@workstation:~$ pip install tempest/
```

```bash
user@workstation:~$ tempest init akraino
2018-03-29 12:43:47.814 22332 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
user@workstation:~$
```

```bash
user@workstation:~$ cd tempest/
user@workstation:~/tempest$ ls
bindep.txt  HACKING.rst  README.rst        REVIEWING.rst  setup.py               tools
doc         LICENSE      releasenotes      roles          tempest                tox.ini
etc         playbooks    requirements.txt  setup.cfg      test-requirements.txt
user@workstation:~/tempest$
```

## Tempest :: Installation :: Configuration

* [github.com/takekawa/tempest\_test](https://github.com/takekawa/tempest_test/blob/master/tempest.conf)

```bash
user@workstation:~/tempest$ sudo apt install tox
user@workstation:~/tempest$
```

```bash
user@workstation:~/tempest$ tox -e genconfig
genconfig create: /home/user/tempest/.tox/genconfig
genconfig installdeps: -chttps://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt, -r/home/user/tempest/requirements.txt, -r/home/user/tempest/test-requirements.txt
genconfig develop-inst: /home/user/tempest
genconfig installed: /home/user/tempest/.tox/genconfig/lib/python3.5/site.py:165: DeprecationWarning: 'U' mode is deprecated,  f = open(fullname, "rU"),appdirs==1.4.3,asn1crypto==0.24.0,Babel==2.5.3,bcrypt==3.1.4,certifi==2018.1.18,cffi==1.11.5,chardet==3.0.4,cliff==2.11.0,cmd2==0.8.2,coverage==4.5.1,cryptography==2.2.2,debtcollector==1.19.0,extras==1.0.0,fasteners==0.14.1,fixtures==3.0.0,flake8==2.5.5,flake8-import-order==0.11,future==0.16.0,hacking==0.12.0,idna==2.6,iso8601==0.1.12,jsonschema==2.6.0,keystoneauth1==3.4.0,linecache2==1.0.0,mccabe==0.2.1,mock==2.0.0,monotonic==1.4,mox3==0.25.0,msgpack==0.5.6,netaddr==0.7.19,netifaces==0.10.6,os-client-config==1.29.0,oslo.concurrency==3.26.0,oslo.config==5.2.0,oslo.context==2.20.0,oslo.i18n==3.20.0,oslo.log==3.37.0,oslo.serialization==2.25.0,oslo.utils==3.36.0,oslotest==3.4.0,paramiko==2.4.1,pbr==3.1.1,pep8==1.5.7,pkg-resources==0.0.0,prettytable==0.7.2,pyasn1==0.4.2,pycodestyle==2.3.1,pycparser==2.18,pyflakes==0.8.1,pyinotify==0.9.6,PyNaCl==1.2.1,pyparsing==2.2.0,pyperclip==1.6.0,python-dateutil==2.7.2,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2018.3,PyYAML==3.12,requests==2.18.4,requestsexceptions==1.4.0,rfc3986==1.1.0,six==1.11.0,stestr==2.0.0,stevedore==1.28.0,-e git+https://git.openstack.org/openstack/tempest@b8b6b051ff382f13db63bac522568f1bde0a2d16#egg=tempest,testrepository==0.0.20,testtools==2.3.0,traceback2==1.4.0,unittest2==1.1.0,urllib3==1.22,voluptuous==0.11.1,wrapt==1.10.11
genconfig runtests: PYTHONHASHSEED='420769913'
genconfig runtests: commands[0] | oslo-config-generator --config-file tempest/cmd/config-generator.tempest.conf
/home/user/tempest/.tox/genconfig/lib/python3.5/site.py:165: DeprecationWarning: 'U' mode is deprecated
  f = open(fullname, "rU")
/home/user/tempest/.tox/genconfig/lib/python3.5/site-packages/oslo_config/generator.py:770: ResourceWarning: unclosed file <_io.TextIOWrapper name='etc/tempest.conf.sample' mode='w' encoding='UTF-8'>
  generate(conf)
_____________________________________________ summary _____________________________________________
  genconfig: commands succeeded
  congratulations :)
user@workstation:~/tempest$
```

```bash
user@workstation:~/tempest$ cp etc/accounts.yaml.sample etc/accounts.yaml
user@workstation:~/tempest$ cp etc/tempest.conf.sample etc/tempest.conf
user@workstation:~/tempest$
```

From OpenStack Dashboard, create 4 new Projects:

* Project: tempestone
* Project: tempesttwo
* Project: tempestthree
* Project: tempestfur

From OpenStack Dashboard, create 4 new Admin Users:

* User: tempestone
  * Primary Project: tempestone
  * Role: admin
* User: tempesttwo
  * Primary Project: tempesttwo
  * Role: admin
* User: tempestthree
  * Primary Project: tempestthree
  * Role: admin
* User: tempestfour
  * Primary Project: tempestfour
  * Role: admin

Some guidelines

* User WITHOUT administrative privileges
* Username of an alternate user WITHOUT administrative privileges
* Username of a user WITH administrative privileges
* [QA Testcase Run Tempest Tests](https://fedoraproject.org/wiki/QA:Testcase_run_Tempest_tests)

```bash
user@workstation:~/tempest$ nano etc/accounts.yaml
```

```bash
- username: 'admin'
  tenant_name: 'admin'
  password: 'admin*'

- username: 'tempestone'
  tenant_name: 'tempestone'
  password: 'admin'

- username: 'tempesttwo'
  tenant_name: 'tempesttwo'
  password: 'admin'

- username: 'tempestthree'
  tenant_name: 'tempestthree'
  password: 'admin'

- username: 'tempestfour'
  tenant_name: 'tempestfour'
  password: 'admin'
```

```
user@workstation:~/tempest$ nano etc/tempest.conf
```

### Type Of Errors

#### One

Timed out

```bash
a0200710>, 'Connection to 192.168.204.2 timed out. (connect timeout=60)')': /v3/domains?name=Default: ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7f1ea0200710>, 'Connection to 192.168.204.2 timed out. (connect timeout=60)')
```

Modify these options

* Timed out
  * admin
  * internal
  * adminURL
  * internalURL
* Pass
  * public
  * publicURL

```bash
# The endpoint type to use for OpenStack Identity (Keystone) API v3.
# The default value adminURL is deprecated and will be modified to
# publicURL in the next release. (string value)
# Possible values:
# public - <No description provided>
# admin - <No description provided>
# internal - <No description provided>
# publicURL - <No description provided>
# adminURL - <No description provided>
# internalURL - <No description provided>
v3_endpoint_type = publicURL
```

#### Two

```bash
    tempest.lib.exceptions.Unauthorized: Unauthorized
    Details: The request you have made requires authentication.
```

Under etc/tempest.conf

```bash
# Password to use for an administrative user. This is needed for
# authenticating requests made by project isolation to create users
# and projects (string value)
admin_password = password
```

```
# Whether keystone identity v3 policy required a domain scoped token
# to use admin APIs (boolean value)
admin_domain_scope = false
```

#### Three

Cannot test one test without account file even if I am using dynamic credentials

```bash
[auth]
#initial#test_accounts_file = /home/user/refstack-client/etc/accounts.yaml
#initial#use_dynamic_credentials = false
use_dynamic_credentials = True
tempest_roles = Member
admin_domain_name = Default
admin_project_name = admin
admin_password = Madawa    
admin_username = admin
```

```bash
(.venv) user@workstation:~/refstack-client$ refstack-client test -c etc/tempest.conf -v -- --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
2018-04-02 10:54:11,855 refstack_client:214 ERROR User credentials cannot be found. User credentials should be defined in the accounts file as described in the Tempest configuration guide (http://docs.openstack.org/developer/tempest/configuration.html).
(.venv) user@workstation:~/refstack-client$
```

If test\_accounts\_file = /home/user/refstack-client/etc/accounts.yaml is enabled, the following message is presented:

```bash
      File "/home/user/refstack-client/.tempest/tempest/lib/services/identity/v3/token_client.py", line 161, in request
        raise exceptions.Unauthorized(resp_body['error']['message'])
    tempest.lib.exceptions.Unauthorized: Unauthorized
    Details: The request you have made requires authentication.
```

#### Tempest :: Akraino :: Run :: Identity

```bash
user@workstation:~/tempest$ tempest run --smoke --list-tests | grep identity | grep v3
tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON.test_get_trusts_all[id-4773ebd5-ecbf-4255-b8d8-b63e6f72b65d,smoke]
tempest.api.identity.admin.v3.test_credentials.CredentialsTestJSON.test_credentials_create_get_update_delete[id-7cd59bf9-bda4-4c72-9467-d21cab278355,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_media_types[id-657c1970-4722-4189-8831-7325f3bc4265,smoke]
tempest.api.identity.admin.v3.test_domains.DomainsTestJSON.test_create_update_delete_domain[id-f2f5b44a-82e8-4dad-8084-0661ea3b18cf,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_list_api_versions[id-721f480f-35b6-46c7-846e-047e6acea0dc,smoke]
tempest.api.identity.admin.v3.test_domains.DefaultDomainTestJSON.test_default_domain_exists[id-17a5de24-e6a0-4e4a-a9ee-d85b6e5612b5,smoke]
tempest.api.identity.admin.v3.test_services.ServicesTestJSON.test_create_update_get_service[id-5193aad5-bcb7-411d-85b0-b3b61b96ef06,smoke]
tempest.api.identity.admin.v3.test_policies.PoliciesTestJSON.test_create_update_delete_policy[id-e544703a-2f03-4cf2-9b0f-350782fdb0d3,smoke]
tempest.api.identity.admin.v3.test_regions.RegionsTestJSON.test_create_region_with_specific_id[id-2c12c5b5-efcf-4aa5-90c5-bff1ab0cdbe2,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_resources[id-b9232f5e-d9e5-4d97-b96c-28d3db4de1bd,smoke]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_role_create_update_show_list[id-18afc6c0-46cf-4911-824e-9989cc056c3a,smoke]
tempest.api.identity.admin.v3.test_endpoints.EndPointsTestJSON.test_update_endpoint[id-37e8f15e-ee7c-4657-a1e7-f6b61e375eff,smoke]
tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_group_users_add_list_delete[id-1598521a-2f36-4606-8df9-30772bd51339,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_statuses[id-8879a470-abfb-47bb-bb8d-5a7fd279ad1e,smoke]
user@workstation:~/tempest$
```

```bash
user@workstation:~/tempest$ nano testslist
tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON.test_get_trusts_all
tempest.api.identity.admin.v3.test_credentials.CredentialsTestJSON.test_credentials_create_get_update_delete
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_media_types
tempest.api.identity.admin.v3.test_domains.DomainsTestJSON.test_create_update_delete_domain
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_list_api_versionstempest.api.identity.admin.v3.test_domains.DefaultDomainTestJSON.test_default_domain_exists
tempest.api.identity.admin.v3.test_services.ServicesTestJSON.test_create_update_get_service
tempest.api.identity.admin.v3.test_policies.PoliciesTestJSON.test_create_update_delete_policy
tempest.api.identity.admin.v3.test_regions.RegionsTestJSON.test_create_region_with_specific_id
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_resources
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_role_create_update_show_list
tempest.api.identity.admin.v3.test_endpoints.EndPointsTestJSON.test_update_endpoint
tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_group_users_add_list_delete
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_statuses
```

```
user@workstation:~/tempest$ tempest run --concurrency 1 --whitelist-file testslist
...
...
======
Totals
======
Ran: 12 tests in 93.0000 sec.
 - Passed: 11
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 1
Sum of execute time for each test: 16.4967 sec.
```

#### Tempest :: Akraino :: Run :: Image

```bash
user@workstation:~/tempest$ tempest run --smoke --list-tests | grep image
tempest.api.volume.test_volumes_get.VolumesGetTest.test_volume_create_get_update_delete_from_image[id-54a01030-c7fc-447c-86ee-c1182beae638,image,smoke]
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image[id-f66891a7-a35c-41a8-b590-a065c2a1caa6,smoke]
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_delete_image[id-f848bb94-1c6e-45a4-8726-39e3a5b23535,smoke]
tempest.api.image.v2.test_versions.VersionsTest.test_list_versions[id-659ea30a-a17c-4317-832c-0f68ed23c31d,smoke]
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_register_upload_get_image_file[id-139b765e-7f3d-4b3d-8b37-3ca3876ee318,smoke]
user@workstation:~/tempest$
```

```bash
user@workstation:~/tempest$ nano testslist
tempest.api.volume.test_volumes_get.VolumesGetTest.test_volume_create_get_update_delete_from_image
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_delete_image
tempest.api.image.v2.test_versions.VersionsTest.test_list_versions
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_register_upload_get_image_file
```

```bash
user@workstation:~/tempest$ tempest run --concurrency 1 --whitelist-file testslist
...
...
======
Totals
======
Ran: 5 tests in 16.0000 sec.
 - Passed: 4
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 1
Sum of execute time for each test: 1.9383 sec.
```

#### Tempest :: Akraino :: Run :: Compute

```bash
user@workstation:~/tempest$ tempest run --smoke --list-tests | grep compute
tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_add_remove_fixed_ip[id-c7e0e60b-ee45-43d0-abeb-8596fd42a2f9,network,smoke]
tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_verify_server_details[id-5de47127-9977-400a-936f-abcfbec1218f,smoke]
tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_list_servers[id-9a438d88-10c6-4bcd-8b5b-5b6e25e1346f,smoke]
tempest.api.compute.test_versions.TestVersions.test_get_version_details[id-b953a29e-929c-4a8e-81be-ec3a7e03cb76,smoke]
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard[id-2cb1baf6-ac8d-4429-bf0d-ba8a0ba53e32,smoke]
tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance[compute,id-fff42874-7db5-4487-a8e1-ddda5fb5288d,smoke]
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_list[id-a6154130-5a55-4850-8be4-5e9e796dbf17,smoke]
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_list_servers[id-9a438d88-10c6-4bcd-8b5b-5b6e25e1346f,smoke]
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses_by_network[id-87bbc374-5538-4f64-b673-2b0e4443cc30,network,smoke]
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_verify_server_details[id-5de47127-9977-400a-936f-abcfbec1218f,smoke]
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke]
tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes[compute,id-9cecbe35-b9d4-48da-a37e-7ce70aa43d30,network,smoke]
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_verify_server_details[id-5de47127-9977-400a-936f-abcfbec1218f,smoke]
tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON.test_security_groups_create_list_delete[id-eb2b087d-633d-4d0d-a7bd-9e6ba35b32de,smoke]
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_list_flavors[id-e36c0eaa-dff5-4082-ad1f-3f9a80aa3f59,smoke]
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses[id-6eb718c0-02d9-4d5e-acd1-4e0c269cef39,network,smoke]
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_get_flavor[id-1f12046b-753d-40d2-abb6-d8eb8b30cb2f,smoke]
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_list_servers[id-9a438d88-10c6-4bcd-8b5b-5b6e25e1346f,smoke]
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_create[id-850795d7-d4d3-4e55-b527-a774c0123d3a,smoke]
tempest.api.compute.test_versions.TestVersions.test_list_api_versions[id-6c0a0990-43b6-4529-9b61-5fd8daf7c55c,smoke]
user@workstation:~/tempest$
```

```bash
user@workstation:~/tempest$ nano testslist
tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_add_remove_fixed_ip
tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_verify_server_details
tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_list_servers
tempest.api.compute.test_versions.TestVersions.test_get_version_details
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard
tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_list
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_list_servers
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses_by_network
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_verify_server_details
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops
tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_verify_server_details
tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON.test_security_groups_create_list_delete
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_list_flavors
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_get_flavor
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_list_servers
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_create
tempest.api.compute.test_versions.TestVersions.test_list_api_versions
```

```bash
user@workstation:~/tempest$ tempest run --concurrency 1 --whitelist-file testlists
...
...
======
Totals
======
Ran: 33 tests in 191.0000 sec.
 - Passed: 27
 - Skipped: 3
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 3
Sum of execute time for each test: 51.9272 sec.
```

#### Tempest :: Akraino :: Run :: List Tests

```bash
user@workstation:~/tempest$ tempest run --smoke --concurrency 1 --serial --list-tests
tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_add_remove_fixed_ip[id-c7e0e60b-ee45-43d0-abeb-8596fd42a2f9,network,smoke]
tempest.api.network.test_security_groups.SecGroupTest.test_list_security_groups[id-e30abd17-fef9-4739-8617-dc26da88e686,smoke]
tempest.api.network.test_networks.NetworksTest.test_list_networks[id-f7ffdeda-e200-4a7a-bcbe-05716e86bf43,smoke]
tempest.api.network.test_ports.PortsTestJSON.test_create_port_in_allowed_allocation_pools[id-0435f278-40ae-48cb-a404-b8a087bc09b1,smoke]
tempest.api.identity.v2.test_api_discovery.TestApiDiscovery.test_api_version_resources[id-ea889a68-a15f-4166-bfb1-c12456eae853,smoke]
tempest.api.network.test_networks.NetworksTest.test_external_network_visibility[id-af774677-42a9-4e4b-bb58-16fe6a5bc1ec,smoke]
tempest.api.object_storage.test_container_services.ContainerTest.test_list_container_contents[id-312ff6bd-5290-497f-bda1-7c5fec6697ab,smoke]
tempest.api.network.test_routers.RoutersTest.test_create_show_list_update_delete_router[id-f64403e2-8483-4b34-8ccd-b09a87bcc68c,smoke]
tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_verify_server_details[id-5de47127-9977-400a-936f-abcfbec1218f,smoke]
tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_list_servers[id-9a438d88-10c6-4bcd-8b5b-5b6e25e1346f,smoke]
tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON.test_get_trusts_all[id-4773ebd5-ecbf-4255-b8d8-b63e6f72b65d,smoke]
tempest.api.object_storage.test_container_quotas.ContainerQuotasTest.test_upload_valid_object[id-9a0fb034-86af-4df0-86fa-f8bd7db21ae0,smoke]
tempest.api.compute.test_versions.TestVersions.test_get_version_details[id-b953a29e-929c-4a8e-81be-ec3a7e03cb76,smoke]
tempest.api.network.test_security_groups.SecGroupTest.test_create_list_update_show_delete_security_group[id-bfd128e5-3c92-44b6-9d66-7fe29d22c802,smoke]
tempest.api.network.test_networks.NetworksTest.test_show_network[id-2bf13842-c93f-4a69-83ed-717d2ec3b44e,smoke]
tempest.api.network.test_networks.NetworksTest.test_create_update_delete_network_subnet[id-0e269138-0da6-4efc-a46d-578161e7b221,smoke]
tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet[id-8936533b-c0aa-4f29-8e53-6cc873aec489,smoke]
tempest.api.volume.test_volumes_get.VolumesGetTest.test_volume_create_get_update_delete_from_image[id-54a01030-c7fc-447c-86ee-c1182beae638,image,smoke]
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image[id-f66891a7-a35c-41a8-b590-a065c2a1caa6,smoke]
tempest.api.network.test_networks.NetworksTest.test_show_subnet[id-bd635d81-6030-4dd1-b3b9-31ba0cfdf6cc,smoke]
tempest.api.identity.admin.v3.test_credentials.CredentialsTestJSON.test_credentials_create_get_update_delete[id-7cd59bf9-bda4-4c72-9467-d21cab278355,smoke]
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_delete_image[id-f848bb94-1c6e-45a4-8726-39e3a5b23535,smoke]
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard[id-2cb1baf6-ac8d-4429-bf0d-ba8a0ba53e32,smoke]
tempest.api.network.test_ports.PortsTestJSON.test_create_update_delete_port[id-c72c1c0c-2193-4aca-aaa4-b1442640f51c,smoke]
tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance[compute,id-fff42874-7db5-4487-a8e1-ddda5fb5288d,smoke]
tempest.api.network.test_ports.PortsTestJSON.test_create_port_with_no_securitygroups[id-4179dcb9-1382-4ced-84fe-1b91c54f5735,smoke]
tempest.api.network.test_networks.NetworksTest.test_list_subnets[id-db68ba48-f4ea-49e9-81d1-e367f6d0b20a,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_media_types[id-657c1970-4722-4189-8831-7325f3bc4265,smoke]
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_list[id-a6154130-5a55-4850-8be4-5e9e796dbf17,smoke]
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_list_servers[id-9a438d88-10c6-4bcd-8b5b-5b6e25e1346f,smoke]
tempest.api.object_storage.test_account_services.AccountTest.test_list_containers[id-3499406a-ae53-4f8c-b43a-133d4dc6fe3f,smoke]
tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list[id-0b6ddd39-b948-471f-8038-4787978747c4,smoke]
tempest.api.object_storage.test_container_services.ContainerTest.test_list_container_metadata[id-96e68f0e-19ec-4aa2-86f3-adc6a45e14dd,smoke]
tempest.api.image.v2.test_versions.VersionsTest.test_list_versions[id-659ea30a-a17c-4317-832c-0f68ed23c31d,smoke]
tempest.api.network.test_subnetpools_extensions.SubnetPoolsTestJSON.test_create_list_show_update_delete_subnetpools[id-62595970-ab1c-4b7f-8fcc-fddfe55e9811,smoke]
tempest.api.identity.admin.v3.test_domains.DomainsTestJSON.test_create_update_delete_domain[id-f2f5b44a-82e8-4dad-8084-0661ea3b18cf,smoke]
tempest.api.object_storage.test_container_quotas.ContainerQuotasTest.test_upload_large_object[id-22eeeb2b-3668-4160-baef-44790f65a5a0,smoke]
tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_port_id[id-2b7d2f37-6748-4d78-92e5-1d590234f0d5,smoke]
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_register_upload_get_image_file[id-139b765e-7f3d-4b3d-8b37-3ca3876ee318,smoke]
tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_show_delete_security_group_rule[id-cfb99e0e-7410-4a3d-8a0c-959a63ee77e9,smoke]
tempest.api.object_storage.test_object_services.ObjectTest.test_get_object[id-02610ba7-86b7-4272-9ed8-aa8d417cb3cd,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_list_api_versions[id-721f480f-35b6-46c7-846e-047e6acea0dc,smoke]
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses_by_network[id-87bbc374-5538-4f64-b673-2b0e4443cc30,network,smoke]
tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_in_allowed_allocation_pools[id-0435f278-40ae-48cb-a404-b8a087bc09b1,smoke]
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_verify_server_details[id-5de47127-9977-400a-936f-abcfbec1218f,smoke]
tempest.api.network.test_routers.RoutersIpV6Test.test_create_show_list_update_delete_router[id-f64403e2-8483-4b34-8ccd-b09a87bcc68c,smoke]
tempest.api.identity.admin.v3.test_domains.DefaultDomainTestJSON.test_default_domain_exists[id-17a5de24-e6a0-4e4a-a9ee-d85b6e5612b5,smoke]
tempest.api.network.test_versions.NetworksApiDiscovery.test_api_version_resources[id-cac8a836-c2e0-4304-b556-cd299c7281d1,smoke]
tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility[id-af774677-42a9-4e4b-bb58-16fe6a5bc1ec,smoke]
tempest.api.object_storage.test_object_services.ObjectTest.test_update_object_metadata[id-7a94c25d-66e6-434c-9c38-97d4e2c29945,smoke]
tempest.api.identity.admin.v3.test_services.ServicesTestJSON.test_create_update_get_service[id-5193aad5-bcb7-411d-85b0-b3b61b96ef06,smoke]
tempest.api.identity.admin.v2.test_services.ServicesTestJSON.test_list_services[id-34ea6489-012d-4a86-9038-1287cadd5eca,smoke]
tempest.api.network.test_networks.BulkNetworkOpsTest.test_bulk_create_delete_network[id-d4f9024d-1e28-4fc1-a6b1-25dbc6fa11e2,smoke]
tempest.api.network.test_networks.NetworksIpV6Test.test_show_subnet[id-bd635d81-6030-4dd1-b3b9-31ba0cfdf6cc,smoke]
tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network[id-d4f9024d-1e28-4fc1-a6b1-25dbc6fa11e2,smoke]
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke]
tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes[compute,id-9cecbe35-b9d4-48da-a37e-7ce70aa43d30,network,smoke]
tempest.api.identity.admin.v3.test_policies.PoliciesTestJSON.test_create_update_delete_policy[id-e544703a-2f03-4cf2-9b0f-350782fdb0d3,smoke]
tempest.api.object_storage.test_account_services.AccountTest.test_list_account_metadata[id-4894c312-6056-4587-8d6f-86ffbf861f80,smoke]
tempest.api.identity.admin.v3.test_regions.RegionsTestJSON.test_create_region_with_specific_id[id-2c12c5b5-efcf-4aa5-90c5-bff1ab0cdbe2,smoke]
tempest.api.network.test_security_groups.SecGroupIPv6Test.test_list_security_groups[id-e30abd17-fef9-4739-8617-dc26da88e686,smoke]
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_verify_server_details[id-5de47127-9977-400a-936f-abcfbec1218f,smoke]
tempest.api.network.test_networks.BulkNetworkOpsTest.test_bulk_create_delete_subnet[id-8936533b-c0aa-4f29-8e53-6cc873aec489,smoke]
tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON.test_security_groups_create_list_delete[id-eb2b087d-633d-4d0d-a7bd-9e6ba35b32de,smoke]
tempest.api.network.test_ports.PortsTestJSON.test_list_ports[id-cf95b358-3e92-4a29-a148-52445e1ac50e,smoke]
tempest.api.network.test_ports.PortsIpV6TestJSON.test_list_ports[id-cf95b358-3e92-4a29-a148-52445e1ac50e,smoke]
tempest.api.network.test_routers.RoutersTest.test_add_multiple_router_interfaces[id-802c73c9-c937-4cef-824b-2191e24a6aab,smoke]
tempest.api.network.test_networks.NetworksIpV6Test.test_create_update_delete_network_subnet[id-0e269138-0da6-4efc-a46d-578161e7b221,smoke]
tempest.api.object_storage.test_object_services.ObjectTest.test_list_object_metadata[id-9a447cf6-de06-48de-8226-a8c6ed31caf2,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_resources[id-b9232f5e-d9e5-4d97-b96c-28d3db4de1bd,smoke]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_role_create_update_show_list[id-18afc6c0-46cf-4911-824e-9989cc056c3a,smoke]
tempest.api.identity.admin.v2.test_users.UsersTestJSON.test_create_user[id-2d55a71e-da1d-4b43-9c03-d269fd93d905,smoke]
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]
tempest.api.network.test_extensions.ExtensionsTestJSON.test_list_show_extensions[id-ef28c7e6-e646-4979-9d67-deb207bc5564,smoke]
tempest.api.object_storage.test_object_services.ObjectTest.test_create_object[id-5b4ce26f-3545-46c9-a2ba-5754358a4c62,smoke]
tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_with_no_securitygroups[id-4179dcb9-1382-4ced-84fe-1b91c54f5735,smoke]
tempest.api.volume.test_versions.VersionsTest.test_list_versions[id-77838fc4-b49b-4c64-9533-166762517369,smoke]
tempest.api.object_storage.test_account_quotas.AccountQuotasTest.test_upload_valid_object[id-a22ef352-a342-4587-8f47-3bbdb5b039c4,smoke]
tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_subnet_id[id-b42e6e39-2e37-49cc-a6f4-8467e940900a,smoke]
tempest.api.network.test_networks.NetworksIpV6Test.test_show_network[id-2bf13842-c93f-4a69-83ed-717d2ec3b44e,smoke]
tempest.api.identity.admin.v3.test_endpoints.EndPointsTestJSON.test_update_endpoint[id-37e8f15e-ee7c-4657-a1e7-f6b61e375eff,smoke]
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_list_flavors[id-e36c0eaa-dff5-4082-ad1f-3f9a80aa3f59,smoke]
tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses[id-6eb718c0-02d9-4d5e-acd1-4e0c269cef39,network,smoke]
tempest.api.network.test_floating_ips.FloatingIPTestJSON.test_create_floating_ip_specifying_a_fixed_ip_address[id-36de4bd0-f09c-43e3-a8e1-1decc1ffd3a5,smoke]
tempest.api.volume.test_volumes_get.VolumesGetTest.test_volume_create_get_update_delete[id-27fb0e9f-fb64-41dd-8bdb-1ffa762f0d51,smoke]
tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_group_users_add_list_delete[id-1598521a-2f36-4606-8df9-30772bd51339,smoke]
tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_get_flavor[id-1f12046b-753d-40d2-abb6-d8eb8b30cb2f,smoke]
tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_update_delete_port[id-c72c1c0c-2193-4aca-aaa4-b1442640f51c,smoke]
tempest.api.object_storage.test_account_quotas.AccountQuotasTest.test_admin_modify_quota[id-63f51f9f-5f1d-4fc6-b5be-d454d70949d6,smoke]
tempest.api.object_storage.test_container_quotas.ContainerQuotasTest.test_upload_too_many_objects[id-3a387039-697a-44fc-a9c0-935de31f426b,smoke]
tempest.api.network.test_ports.PortsTestJSON.test_show_port[id-c9a685bd-e83f-499c-939f-9f7863ca259f,smoke]
tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_port_id[id-2b7d2f37-6748-4d78-92e5-1d590234f0d5,smoke]
tempest.api.identity.v2.test_api_discovery.TestApiDiscovery.test_api_version_statuses[id-77fd6be0-8801-48e6-b9bf-38cdd2f253ec,smoke]
tempest.api.compute.servers.test_create_server.ServersTestJSON.test_list_servers[id-9a438d88-10c6-4bcd-8b5b-5b6e25e1346f,smoke]
tempest.api.object_storage.test_container_services.ContainerTest.test_create_container[id-92139d73-7819-4db1-85f8-3f2f22a8d91f,smoke]
tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_create[id-850795d7-d4d3-4e55-b527-a774c0123d3a,smoke]
tempest.api.network.test_security_groups.SecGroupTest.test_create_show_delete_security_group_rule[id-cfb99e0e-7410-4a3d-8a0c-959a63ee77e9,smoke]
tempest.api.network.test_routers.RoutersIpV6Test.test_add_multiple_router_interfaces[id-802c73c9-c937-4cef-824b-2191e24a6aab,smoke]
tempest.api.network.test_floating_ips.FloatingIPTestJSON.test_create_list_show_update_delete_floating_ip[id-62595970-ab1c-4b7f-8fcc-fddfe55e8718,smoke]
tempest.api.compute.test_versions.TestVersions.test_list_api_versions[id-6c0a0990-43b6-4529-9b61-5fd8daf7c55c,smoke]
tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_subnet_id[id-b42e6e39-2e37-49cc-a6f4-8467e940900a,smoke]
tempest.api.network.test_networks.BulkNetworkOpsTest.test_bulk_create_delete_port[id-48037ff2-e889-4c3b-b86a-8e3f34d2d060,smoke]
tempest.api.network.test_ports.PortsIpV6TestJSON.test_show_port[id-c9a685bd-e83f-499c-939f-9f7863ca259f,smoke]
tempest.api.identity.v2.test_api_discovery.TestApiDiscovery.test_api_media_types[id-007a0be0-78fe-4fdb-bbee-e9216cc17bb2,smoke]
tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_list_update_show_delete_security_group[id-bfd128e5-3c92-44b6-9d66-7fe29d22c802,smoke]
tempest.api.network.test_networks.NetworksIpV6Test.test_list_subnets[id-db68ba48-f4ea-49e9-81d1-e367f6d0b20a,smoke]
tempest.api.network.test_networks.NetworksIpV6Test.test_list_networks[id-f7ffdeda-e200-4a7a-bcbe-05716e86bf43,smoke]
tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_statuses[id-8879a470-abfb-47bb-bb8d-5a7fd279ad1e,smoke]
tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port[id-48037ff2-e889-4c3b-b86a-8e3f34d2d060,smoke]
user@workstation:~/tempest$
```

### Tempest :: Akraino :: Run :: nosetests

> In Process

```bash
user@workstation:~/tempest$ nosetests -sv tempest.scenario.test_minimum_basic
2018-03-29 13:20:48.647 25588 INFO tempest [-] Using tempest config file /home/user/tempest/etc/tempest.conf
2018-03-29 13:20:49.440 25588 INFO tempest.lib.common.rest_client [req-4b67138f-176e-440c-b11a-9b1031672f8f ] Request (TestMinimumBasicScenario:setUpClass): 201 POST http://10.10.10.2:5000/v3/auth/tokens
2018-03-29 13:20:50.191 25588 INFO tempest.lib.common.rest_client [req-01a5bfa4-cbaf-4b50-b0d5-88a699e4702f ] Request (TestMinimumBasicScenario:setUpClass): 201 POST http://10.10.10.2:5000/v3/auth/tokens
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://theiotlearninginitiative.gitbook.io/edgecomputingsolutions/introduction/stacks/openstack/testing/akraino/tempest.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.