OpenStack is, undoubtedly, a really huge ecosystem of cooperative services. Rally is a benchmarking tool that answers the question: "How does OpenStack work at scale?". To make this possible, Rally automates and unifies multi-node OpenStack deployment, cloud verification, benchmarking & profiling. Rally does it in a generic way, making it possible to check whether OpenStack is going to work well on, say, a 1k-servers installation under high load. Thus it can be used as a basic tool for an OpenStack CI/CD system that would continuously improve its SLA, performance and stability.
Tempest
Tempest Testing Project
Other
(openstack) module list
user@workstation:~/$ mkdir ma
user@workstation:~/$ cd ma
user@workstation:~/ma$ virtualenv titaniumcloud
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/user/ma/titaniumcloud/bin/python2
Also creating executable in /home/user/ma/titaniumcloud/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
user@workstation:~/ma$
(.venv) user@workstation:~/refstack-client$ refstack-client test -c ./etc/tempest.conf -v -- --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
(.venv) user@workstation:~/refstack-client$ refstack-client test -c ./etc/tempest.conf -v -- --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
2018-03-28 15:14:56,497 refstack_client:139 DEBUG The .testrepository/next-stream file was not found. Assuming subunit results will be stored in file 0.
2018-03-28 15:14:57,266 refstack_client:421 INFO Starting Tempest test...
2018-03-28 15:14:57.845 10230 INFO tempest [-] Using tempest config file /home/user/refstack-client/etc/tempest.conf
2018-03-28 15:14:57.854 10230 WARNING oslo_config.cfg [-] Option "heat" from group "service_available" is deprecated for removal (Heat support will be removed from Tempest). Its value may be silently ignored in the future.
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --list
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --load-list /tmp/tmp5daxWM
2018-03-28 15:14:59.991 10240 INFO tempest [-] Using tempest config file /home/user/refstack-client/etc/tempest.conf
2018-03-28 15:15:00.000 10240 WARNING oslo_config.cfg [-] Option "heat" from group "service_available" is deprecated for removal (Heat support will be removed from Tempest). Its value may be silently ignored in the future.
2018-03-28 15:15:00.337 10240 INFO tempest.lib.common.preprov_creds [-] TokensV3Test allocated creds:
{'username': 'admin', 'tenant_name': 'admin'}
2018-03-28 15:15:00.337 10240 WARNING tempest.lib.common.preprov_creds [-] Using credentials {'username': 'admin', 'tenant_name': 'admin'} for v3 API calls. Assuming they are valid in the default domain.
2018-03-28 15:15:01.074 10240 INFO tempest.lib.common.rest_client [req-8946e52d-9930-4136-a338-340c1fc36935 ] Request (TokensV3Test:setUpClass): 201 POST http://10.10.10.2:5000/v3/auth/tokens
2018-03-28 15:15:01.534 10240 INFO tempest.lib.common.rest_client [req-ffcb7403-27ce-425e-8331-824e61a28687 ] Request (TokensV3Test:test_create_token): 201 POST http://10.10.10.2:5000/v3/auth/tokens
{0} tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token [0.460717s] ... ok
2018-03-28 15:15:01.545 10240 INFO tempest.lib.common.preprov_creds [-] TokensV3Test returned allocated creds:
{'username': 'admin', 'tenant_name': 'admin', 'domain_name': 'Default'}
======
Totals
======
Ran: 1 tests in 3.0000 sec.
- Passed: 1
- Skipped: 0
- Expected Fail: 0
- Unexpected Success: 0
- Failed: 0
Sum of execute time for each test: 0.4607 sec.
==============
Worker Balance
==============
- Worker 0 (1 tests) => 0:00:00.460717
2018-03-28 15:15:01,686 refstack_client:481 INFO Tempest test complete.
2018-03-28 15:15:01,686 refstack_client:482 INFO Subunit results located in: /home/user/refstack-client/.tempest/.testrepository/0
2018-03-28 15:15:01,688 refstack_client:485 INFO Number of passed tests: 1
2018-03-28 15:15:01,688 refstack_client:497 INFO JSON results saved in: /home/user/refstack-client/.tempest/.testrepository/0.json
(.venv) user@workstation:~/refstack-client$
REfStack :: Identity
OpenStack Interoperability. RefStack is a source of tools for OpenStack interoperability testing.
Refstack Design RefStack provides the Interop Working Group, a.k.a. DefCore committee, the tools for vendors and other users to run API tests against their clouds to provide the Interop Working Group with a reliable overview of what APIs and capabilities are being used in the marketplace.