Testing

Testing

• 1

• 2

Sphinx

• 1

Rally

OpenStack is, undoubtedly, a really huge ecosystem of cooperative services. Rally is a benchmarking tool that answers the question: "How does OpenStack work at scale?". To make this possible, Rally automates and unifies multi-node OpenStack deployment, cloud verification, benchmarking & profiling. Rally does it in a generic way, making it possible to check whether OpenStack is going to work well on, say, a 1k-servers installation under high load. Thus it can be used as a basic tool for an OpenStack CI/CD system that would continuously improve its SLA, performance and stability. Some Webpage

• http://thegeekyway.com/benchmarking-openstack-rally/

Tempest

Tempest Testing Project Homepage

Other

• https://devcentral.f5.com/articles/tempest-21919

(openstack) module list

user@workstation:~/$ mkdir ma
user@workstation:~/$ cd ma
user@workstation:~/ma$ virtualenv titaniumcloud
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/user/ma/titaniumcloud/bin/python2
Also creating executable in /home/user/ma/titaniumcloud/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
user@workstation:~/ma$

user@workstation:~/ma$ source titaniumcloud/bin/activate
(titaniumcloud) user@workstation:~/ma$

(titaniumcloud) user@workstation:~/ma$ git clone https://git.openstack.org/openstack/tempest
Cloning into 'tempest'...
(titaniumcloud) user@workstation:~/ma$ pip install tempest/

(titaniumcloud) user@workstation:~/ma$ ls titaniumcloud/etc/tempest/
accounts.yaml.sample  logging.conf.sample  whitelist.yaml
(titaniumcloud) user@workstation:~/ma$ export TEMPEST_CONFIG_DIR=titaniumcloud/etc/tempest/
(titaniumcloud) user@workstation:~/ma$

• http://git.openstack.org/cgit/openstack/python-tempestconf/tree/etc/default-overrides.conf

(titaniumcloud) user@workstation:~/ma$ tempest account-generator -c titaniumcloud/etc/tempest/tempest.conf --os-username admin --os-password Madawaska1* --os-project-name admin --os-tenant-name admin --os-domain-name Default -r 1 --with-admin -i 3 accounts_file.yaml

(titaniumcloud) user@workstation:~/ma$ easy_install tox
(titaniumcloud) user@workstation:~/ma$ cd tempest/
(titaniumcloud) user@workstation:~/ma/tempest$ tox -epy27 --notest
(titaniumcloud) user@workstation:~/ma/tempest$ source .tox/all-plugin/bin/activate

Old

(.venv) user@workstation:~/$ git clone https://git.openstack.org/openstack/tempest
(.venv) user@workstation:~/$ cd tempest
(.venv) user@workstation:~/tempest$ cd tempest
(.venv) user@workstation:~/tempest$ pip install .

(.venv) user@workstation:~/tempest$ mkdir titaniumcloud
(.venv) user@workstation:~/tempest$ cd titaniumcloud/

(.venv) user@workstation:~/tempest/titaniumcloud$ tempest init
2018-03-28 10:21:42.910 16597 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
(.venv) user@workstation:~/tempest/titaniumcloud$

(.venv) user@workstation:~/tempest/titaniumcloud$ ls
etc  logs  tempest_lock
(.venv) user@workstation:~/tempest/titaniumcloud$

(.venv) user@workstation:~/tempest/titaniumcloud$ tempest account-generator -h

(.venv) user@workstation:~/tempest/titaniumcloud$ tempest account-generator -c /home/user/tempest/titaniumcloud/etc/tempest.conf --os-username admin --os-password Madawaska1* --os-project-name admin --os-tenant-name admin --os-domain-name Default -r 1 --with-admin -i 3 accounts_file.yaml

(.venv) user@workstation:~/tempest/titaniumcloud$ sudo apt install python-testrepository

(.venv) user@workstation:~/tempest/titaniumcloud$ tempest verify-config -o etc/tempest.conf

Other

• https://devcentral.f5.com/articles/tempest-21919

user@workstation:~$ sudo apt install python-pip
user@workstation:~$ virtualenv titaniumcloud
user@workstation:~$ source titaniumcloud/bin/activate
(titaniumcloud) user@workstation:~$

(titaniumcloud) user@workstation:~$ pip install tempest/

(titaniumcloud) user@workstation:~$ export TEMPEST_CONFIG_DIR=titaniumcloud/etc/tempest/
``

# Commands

```sh
(.venv) user@workstation:~/tempest$ tempest workspace list
2018-03-28 11:06:55.278 21000 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
+---------------+----------------------------------+
| Name          | Path                             |
+---------------+----------------------------------+
| cloud-01      | /home/user/cloud-01              |
| titaniumcloud | /home/user/tempest/titaniumcloud |
+---------------+----------------------------------+
(.venv) user@workstation:~/tempest$

(.venv) user@workstation:~/tempest$ tempest list-plugins
+------+------------+
| Name | EntryPoint |
+------+------------+
+------+------------+

Refstack

OpenStack Interoperability. RefStack is a source of tools for OpenStack interoperability testing. Homepage

Refstack Design RefStack provides the Interop Working Group, a.k.a. DefCore committee, the tools for vendors and other users to run API tests against their clouds to provide the Interop Working Group with a reliable overview of what APIs and capabilities are being used in the marketplace. Wiki

user@workstation:~/$ git clone https://github.com/openstack/refstack-client
user@workstation:~/$ cd refstack-client
user@workstation:~/refstack-client$ ./setup_env

user@workstation:~/refstack-client$ find . -name tempest.conf.sample
./etc/tempest.conf.sample
user@workstation:~/refstack-client$ find . -name accounts.yaml.sample
./etc/accounts.yaml.sample
./.tempest/etc/accounts.yaml.sample
./.tempest/.venv/etc/tempest/accounts.yaml.sample
user@workstation:~/refstack-client$

user@workstation:~/refstack-client$ cp ./etc/accounts.yaml.sample ./etc/accounts.yaml
user@workstation:~/refstack-client$ cp ./etc/tempest.conf.sample ./etc/tempest.conf
user@workstation:~/refstack-client$

user@workstation:~/refstack-client$ nano ./etc/accounts.yaml

user@workstation:~/refstack-client$ nano ./etc/tempest.conf

user@workstation:~/refstack-client$ source .venv/bin/activate
(.venv) user@workstation:~/refstack-client$

(.venv) user@workstation:~/refstack-client$ refstack-client test -c ./etc/tempest.conf -v -- --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token

(.venv) user@workstation:~/refstack-client$ refstack-client test -c ./etc/tempest.conf -v -- --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
2018-03-28 15:14:56,497 refstack_client:139 DEBUG The .testrepository/next-stream file was not found. Assuming subunit results will be stored in file 0.
2018-03-28 15:14:57,266 refstack_client:421 INFO Starting Tempest test...
2018-03-28 15:14:57.845 10230 INFO tempest [-] Using tempest config file /home/user/refstack-client/etc/tempest.conf
2018-03-28 15:14:57.854 10230 WARNING oslo_config.cfg [-] Option "heat" from group "service_available" is deprecated for removal (Heat support will be removed from Tempest).  Its value may be silently ignored in the future.
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --list 
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover}  --load-list /tmp/tmp5daxWM
2018-03-28 15:14:59.991 10240 INFO tempest [-] Using tempest config file /home/user/refstack-client/etc/tempest.conf
2018-03-28 15:15:00.000 10240 WARNING oslo_config.cfg [-] Option "heat" from group "service_available" is deprecated for removal (Heat support will be removed from Tempest).  Its value may be silently ignored in the future.
2018-03-28 15:15:00.337 10240 INFO tempest.lib.common.preprov_creds [-] TokensV3Test allocated creds:
{'username': 'admin', 'tenant_name': 'admin'}
2018-03-28 15:15:00.337 10240 WARNING tempest.lib.common.preprov_creds [-] Using credentials {'username': 'admin', 'tenant_name': 'admin'} for v3 API calls. Assuming they are valid in the default domain.
2018-03-28 15:15:01.074 10240 INFO tempest.lib.common.rest_client [req-8946e52d-9930-4136-a338-340c1fc36935 ] Request (TokensV3Test:setUpClass): 201 POST http://10.10.10.2:5000/v3/auth/tokens
2018-03-28 15:15:01.534 10240 INFO tempest.lib.common.rest_client [req-ffcb7403-27ce-425e-8331-824e61a28687 ] Request (TokensV3Test:test_create_token): 201 POST http://10.10.10.2:5000/v3/auth/tokens
{0} tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token [0.460717s] ... ok
2018-03-28 15:15:01.545 10240 INFO tempest.lib.common.preprov_creds [-] TokensV3Test returned allocated creds:
{'username': 'admin', 'tenant_name': 'admin', 'domain_name': 'Default'}

======
Totals
======
Ran: 1 tests in 3.0000 sec.
 - Passed: 1
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 0.4607 sec.

==============
Worker Balance
==============
 - Worker 0 (1 tests) => 0:00:00.460717
2018-03-28 15:15:01,686 refstack_client:481 INFO Tempest test complete.
2018-03-28 15:15:01,686 refstack_client:482 INFO Subunit results located in: /home/user/refstack-client/.tempest/.testrepository/0
2018-03-28 15:15:01,688 refstack_client:485 INFO Number of passed tests: 1
2018-03-28 15:15:01,688 refstack_client:497 INFO JSON results saved in: /home/user/refstack-client/.tempest/.testrepository/0.json
(.venv) user@workstation:~/refstack-client$

REfStack :: Identity

• How to configure and run Tempest for testing OpenStack Keystone Identity V3 API?