settingsLogin | Registersettings

[openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

0 votes

Hi All,

This is a follow up for OpenStack Extreme Testing session[1]
we did in MEX-ops-meetup.

Quick intro for those who were not there:
In this work, we proposed to add new testing framework for openstack.
This framework will provides tool for create tests with destructive
scenarios which will check for High Availability, failover and
recovery of OpenStack cloud.
Please refer the link on top of the [1] for further details.

Follow up:
We are planning periodic irc meeting and have an irc
channel for discussion. I will get back to you with those details soon.

At that session, we did not have time to discuss last 3 items,
Reference architectures
We are discussing about the reference architecture in [2].

What sort of failures do you see today in your environment?
Currently we are considering, service failures, backend services (mq,
DB, etc.) failures,
Network sw failures..etc. To begin with the implementation, we are
considering to start with
service failures. Please let us know what failures are more frequent
in your environment.

Emulation/Simulation mechanisms, etc.
Rather than doing actual scale, load, or performance tests, we are
thinking to build a emulation/simulation mechanism
to get the predictions or result of how will openstack behave on such
situations.
This interesting idea was proposed by the Gautam and need more
discussion on this.

Please let us know you questions or comments.

Request to Mike Perez:
We discussed about synergies with openstack assertion tags and other
efforts to do similar testing in openstack.
Could you please give some info or pointer of previous discussions.

[1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
[2] https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/15477787/Extreme+Testing-Vision+Arch

--- Regards,
Sampath


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Nov 1, 2017 in openstack-dev by Sam_P (1,740 points)   3

7 Responses

0 votes

Sam,

Seems like a good plan and huge topic ;)

I would as well suggest to take a look at the similar efforts in OpenStack:
- Failure injection: https://github.com/openstack/os-faults
- Rally Hooks Mechanism (to inject in rally scenarios failures):
https://rally.readthedocs.io/en/latest/plugins/implementation/hook_and_trigger_plugins.html

Best regards,
Boris Pavlovic

On Mon, Aug 14, 2017 at 2:35 AM, Sam P sam47priya@gmail.com wrote:

Hi All,

This is a follow up for OpenStack Extreme Testing session[1]
we did in MEX-ops-meetup.

Quick intro for those who were not there:
In this work, we proposed to add new testing framework for openstack.
This framework will provides tool for create tests with destructive
scenarios which will check for High Availability, failover and
recovery of OpenStack cloud.
Please refer the link on top of the [1] for further details.

Follow up:
We are planning periodic irc meeting and have an irc
channel for discussion. I will get back to you with those details soon.

At that session, we did not have time to discuss last 3 items,
Reference architectures
We are discussing about the reference architecture in [2].

What sort of failures do you see today in your environment?
Currently we are considering, service failures, backend services (mq,
DB, etc.) failures,
Network sw failures..etc. To begin with the implementation, we are
considering to start with
service failures. Please let us know what failures are more frequent
in your environment.

Emulation/Simulation mechanisms, etc.
Rather than doing actual scale, load, or performance tests, we are
thinking to build a emulation/simulation mechanism
to get the predictions or result of how will openstack behave on such
situations.
This interesting idea was proposed by the Gautam and need more
discussion on this.

Please let us know you questions or comments.

Request to Mike Perez:
We discussed about synergies with openstack assertion tags and other
efforts to do similar testing in openstack.
Could you please give some info or pointer of previous discussions.

[1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
[2] https://openstack-lcoo.atlassian.net/wiki/spaces/
LCOO/pages/15477787/Extreme+Testing-Vision+Arch

--- Regards,
Sampath


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 14, 2017 by boris_at_pavlovic.me (6,900 points)   1 4 6
0 votes

On Mon, Aug 14, 2017 at 7:56 PM, Boris Pavlovic boris@pavlovic.me wrote:
Sam,

Seems like a good plan and huge topic ;)

I would as well suggest to take a look at the similar efforts in OpenStack:
- Failure injection: https://github.com/openstack/os-faults

Yea, we are considering this lib, more details in this spec -
https://review.openstack.org/#/c/443504/

Best regards,
Boris Pavlovic

On Mon, Aug 14, 2017 at 2:35 AM, Sam P sam47priya@gmail.com wrote:

Hi All,

This is a follow up for OpenStack Extreme Testing session[1]
we did in MEX-ops-meetup.

Quick intro for those who were not there:
In this work, we proposed to add new testing framework for openstack.
This framework will provides tool for create tests with destructive
scenarios which will check for High Availability, failover and
recovery of OpenStack cloud.
Please refer the link on top of the [1] for further details.

Follow up:
We are planning periodic irc meeting and have an irc
channel for discussion. I will get back to you with those details soon.

Thanks SamP for updates.
Or another option is use qa channel itself as it is going to be under
qa projects. Similar to patrole etc.

Anyways I have added this in QA PTG agenda [1] to discuss the
architecture and how we can help on this to speedup this.

At that session, we did not have time to discuss last 3 items,
Reference architectures
We are discussing about the reference architecture in [2].

What sort of failures do you see today in your environment?
Currently we are considering, service failures, backend services (mq,
DB, etc.) failures,
Network sw failures..etc. To begin with the implementation, we are
considering to start with
service failures. Please let us know what failures are more frequent
in your environment.

Emulation/Simulation mechanisms, etc.
Rather than doing actual scale, load, or performance tests, we are
thinking to build a emulation/simulation mechanism
to get the predictions or result of how will openstack behave on such
situations.
This interesting idea was proposed by the Gautam and need more
discussion on this.

Please let us know you questions or comments.

Request to Mike Perez:
We discussed about synergies with openstack assertion tags and other
efforts to do similar testing in openstack.
Could you please give some info or pointer of previous discussions.

[1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
[2]
https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/15477787/Extreme+Testing-Vision+Arch

--- Regards,
Sampath

..1 https://etherpad.openstack.org/p/qa-queens-ptg

-gmann


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 14, 2017 by GHANSHYAM_MANN (5,700 points)   1 3 3
0 votes

2017-08-14 13:41 GMT+02:00 Ghanshyam Mann ghanshyammann@gmail.com:

On Mon, Aug 14, 2017 at 7:56 PM, Boris Pavlovic boris@pavlovic.me wrote:

Sam,

Seems like a good plan and huge topic ;)

I would as well suggest to take a look at the similar efforts in
OpenStack:
- Failure injection: https://github.com/openstack/os-faults

Yea, we are considering this lib, more details in this spec -
https://review.openstack.org/#/c/443504/

As a little demonstration of Rally and os-faults working together:
OpenStack reliability tests from performance documentation:
* test plan:
https://docs.openstack.org/performance-docs/latest/test_plans/reliability/version_2/plan.html#reliability-testing-version-2
* example of report:
https://docs.openstack.org/performance-docs/latest/test_results/reliability/version_2/reports/neutron/create_and_list_networks_with_kill_mysql_service_on_one_node/index.html

Best regards,
Ilya Shakhat


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 14, 2017 by shakhat_at_gmail.com (260 points)  
0 votes

+1 for Boris’ suggestion. Many of us use Rally to probe our clouds and have significant tooling behind it to integrate with local availability reporting and trouble ticketing systems. It would be much easier to deploy new functionality such as you propose if it was integrated into an existing project framework (such as Rally).

Tim

From: Boris Pavlovic boris@pavlovic.me
Reply-To: "OpenStack Development Mailing List (not for usage questions)" openstack-dev@lists.openstack.org
Date: Monday, 14 August 2017 at 12:57
To: "OpenStack Development Mailing List (not for usage questions)" openstack-dev@lists.openstack.org
Cc: openstack-operators openstack-operators@lists.openstack.org
Subject: Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

Sam,

Seems like a good plan and huge topic ;)

I would as well suggest to take a look at the similar efforts in OpenStack:
- Failure injection: https://github.com/openstack/os-faults
- Rally Hooks Mechanism (to inject in rally scenarios failures): https://rally.readthedocs.io/en/latest/plugins/implementation/hook_and_trigger_plugins.html

Best regards,
Boris Pavlovic

On Mon, Aug 14, 2017 at 2:35 AM, Sam P sam47priya@gmail.com wrote:
Hi All,

This is a follow up for OpenStack Extreme Testing session[1]
we did in MEX-ops-meetup.

Quick intro for those who were not there:
In this work, we proposed to add new testing framework for openstack.
This framework will provides tool for create tests with destructive
scenarios which will check for High Availability, failover and
recovery of OpenStack cloud.
Please refer the link on top of the [1] for further details.

Follow up:
We are planning periodic irc meeting and have an irc
channel for discussion. I will get back to you with those details soon.

At that session, we did not have time to discuss last 3 items,
Reference architectures
We are discussing about the reference architecture in [2].

What sort of failures do you see today in your environment?
Currently we are considering, service failures, backend services (mq,
DB, etc.) failures,
Network sw failures..etc. To begin with the implementation, we are
considering to start with
service failures. Please let us know what failures are more frequent
in your environment.

Emulation/Simulation mechanisms, etc.
Rather than doing actual scale, load, or performance tests, we are
thinking to build a emulation/simulation mechanism
to get the predictions or result of how will openstack behave on such
situations.
This interesting idea was proposed by the Gautam and need more
discussion on this.

Please let us know you questions or comments.

Request to Mike Perez:
We discussed about synergies with openstack assertion tags and other
efforts to do similar testing in openstack.
Could you please give some info or pointer of previous discussions.

[1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
[2] https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/15477787/Extreme+Testing-Vision+Arch

--- Regards,
Sampath


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 14, 2017 by Tim_Bell (16,440 points)   1 4 8
0 votes

Hi All,

Sending out a gentle reminder of Sydney Summit Forum Session
regarding this topic.

Extreme/Destructive Testing
Tuesday, November 7, 1:50pm-2:30pm
Sydney Convention and Exhibition Centre - Level 4 - C4.11
[https://www.openstack.org/summit/sydney-2017/summit-schedule/events/20470/extremedestructive-testing]
Eatherpad: https://etherpad.openstack.org/p/SYD-extreme-testing

Your participation in this session would be greatly appreciated.
--- Regards,
Sampath

On Mon, Aug 14, 2017 at 11:43 PM, Tim Bell Tim.Bell@cern.ch wrote:
+1 for Boris’ suggestion. Many of us use Rally to probe our clouds and have
significant tooling behind it to integrate with local availability reporting
and trouble ticketing systems. It would be much easier to deploy new
functionality such as you propose if it was integrated into an existing
project framework (such as Rally).

Tim

From: Boris Pavlovic boris@pavlovic.me
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
openstack-dev@lists.openstack.org
Date: Monday, 14 August 2017 at 12:57
To: "OpenStack Development Mailing List (not for usage questions)"
openstack-dev@lists.openstack.org
Cc: openstack-operators openstack-operators@lists.openstack.org
Subject: Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme
Testing

Sam,

Seems like a good plan and huge topic ;)

I would as well suggest to take a look at the similar efforts in OpenStack:

Best regards,
Boris Pavlovic

On Mon, Aug 14, 2017 at 2:35 AM, Sam P sam47priya@gmail.com wrote:

Hi All,

This is a follow up for OpenStack Extreme Testing session[1]
we did in MEX-ops-meetup.

Quick intro for those who were not there:
In this work, we proposed to add new testing framework for openstack.
This framework will provides tool for create tests with destructive
scenarios which will check for High Availability, failover and
recovery of OpenStack cloud.
Please refer the link on top of the [1] for further details.

Follow up:
We are planning periodic irc meeting and have an irc
channel for discussion. I will get back to you with those details soon.

At that session, we did not have time to discuss last 3 items,
Reference architectures
We are discussing about the reference architecture in [2].

What sort of failures do you see today in your environment?
Currently we are considering, service failures, backend services (mq,
DB, etc.) failures,
Network sw failures..etc. To begin with the implementation, we are
considering to start with
service failures. Please let us know what failures are more frequent
in your environment.

Emulation/Simulation mechanisms, etc.
Rather than doing actual scale, load, or performance tests, we are
thinking to build a emulation/simulation mechanism
to get the predictions or result of how will openstack behave on such
situations.
This interesting idea was proposed by the Gautam and need more
discussion on this.

Please let us know you questions or comments.

Request to Mike Perez:
We discussed about synergies with openstack assertion tags and other
efforts to do similar testing in openstack.
Could you please give some info or pointer of previous discussions.

[1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
[2]
https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/15477787/Extreme+Testing-Vision+Arch

--- Regards,
Sampath


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Nov 1, 2017 by Sam_P (1,740 points)   3
0 votes

Sam,

Etherpad: https://etherpad.openstack.org/p/SYD-extreme-testing

I really don't want to sound like a person that say use Rally my best ever
project blablbab and other BS.
I think that "reinventing wheels" approach is how humanity evolves and
that's why I like this effort in any case.

But really, I read carefully etherpad and I really see in description of
Eris just plain Rally as is:

  • Rally allows you to create tests as YAML
  • Rally allows you to inject in various actions during the load (Rally
    Hooks) which makes it easy to do chaos and other kind of testing
  • Rally is pluggable and you can write even your own Runners (scenario
    executors) that will generate load pattern that you need
  • Rally has SLAs plugins (that can deeply analyze result of test cases) and
    say whatever they pass or not
  • We are working on feature that allows you to mix different workloads in
    parallel (and generate more realistic load)
    -.....

So it would be really nice if you can share gaps that you faced that are
blocking you to use directly Rally..

Thanks!

Best regards,
Boris Pavlovic

On Tue, Oct 31, 2017 at 10:50 PM, Sam P sam47priya@gmail.com wrote:

Hi All,

Sending out a gentle reminder of Sydney Summit Forum Session
regarding this topic.

Extreme/Destructive Testing
Tuesday, November 7, 1:50pm-2:30pm
Sydney Convention and Exhibition Centre - Level 4 - C4.11
[https://www.openstack.org/summit/sydney-2017/summit-
schedule/events/20470/extremedestructive-testing]
Eatherpad: https://etherpad.openstack.org/p/SYD-extreme-testing

Your participation in this session would be greatly appreciated.
--- Regards,
Sampath

On Mon, Aug 14, 2017 at 11:43 PM, Tim Bell Tim.Bell@cern.ch wrote:

+1 for Boris’ suggestion. Many of us use Rally to probe our clouds and
have
significant tooling behind it to integrate with local availability
reporting
and trouble ticketing systems. It would be much easier to deploy new
functionality such as you propose if it was integrated into an existing
project framework (such as Rally).

Tim

From: Boris Pavlovic boris@pavlovic.me
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
openstack-dev@lists.openstack.org
Date: Monday, 14 August 2017 at 12:57
To: "OpenStack Development Mailing List (not for usage questions)"
openstack-dev@lists.openstack.org
Cc: openstack-operators openstack-operators@lists.openstack.org
Subject: Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme
Testing

Sam,

Seems like a good plan and huge topic ;)

I would as well suggest to take a look at the similar efforts in
OpenStack:

Best regards,
Boris Pavlovic

On Mon, Aug 14, 2017 at 2:35 AM, Sam P sam47priya@gmail.com wrote:

Hi All,

This is a follow up for OpenStack Extreme Testing session[1]
we did in MEX-ops-meetup.

Quick intro for those who were not there:
In this work, we proposed to add new testing framework for openstack.
This framework will provides tool for create tests with destructive
scenarios which will check for High Availability, failover and
recovery of OpenStack cloud.
Please refer the link on top of the [1] for further details.

Follow up:
We are planning periodic irc meeting and have an irc
channel for discussion. I will get back to you with those details soon.

At that session, we did not have time to discuss last 3 items,
Reference architectures
We are discussing about the reference architecture in [2].

What sort of failures do you see today in your environment?
Currently we are considering, service failures, backend services (mq,
DB, etc.) failures,
Network sw failures..etc. To begin with the implementation, we are
considering to start with
service failures. Please let us know what failures are more frequent
in your environment.

Emulation/Simulation mechanisms, etc.
Rather than doing actual scale, load, or performance tests, we are
thinking to build a emulation/simulation mechanism
to get the predictions or result of how will openstack behave on such
situations.
This interesting idea was proposed by the Gautam and need more
discussion on this.

Please let us know you questions or comments.

Request to Mike Perez:
We discussed about synergies with openstack assertion tags and other
efforts to do similar testing in openstack.
Could you please give some info or pointer of previous discussions.

[1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
[2]
https://openstack-lcoo.atlassian.net/wiki/spaces/
LCOO/pages/15477787/Extreme+Testing-Vision+Arch

--- Regards,
Sampath



OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:
unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:
unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 1, 2017 by boris_at_pavlovic.me (6,900 points)   1 4 6
0 votes

Hi Boris,

Thanks for comment.
Sorry for the misleading context in the etherpad. It sounds like Eris
will do everything, but it is not.
I think I need to add more details to etherpad (I will do that)
Short answer is we are not into "reinventing wheels" and we will use
all possible existing tools.
# Me and Gautam are working on a document about what are the gaps,
and what tool should use on what purpose and why.
# I think that would be the long answer and we definitely need your
feed back on this.
# I will share that here..

For, Rally,
Actually we are using Rally in Eris. Here is the PoC we are working on (WIP).
# At this point, PoC is in it's very early stage. Not all the
Concepts are implemented yet..
code: https://github.com/gautamdivgi/eris
doc: https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/22872097/Extreme+Testing+Demo

You may find lot more doc's about Eris in here,
https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/13393034/Eris+-+Extreme+Testing+Framework+for+OpenStack

For an example, we are considering,
Rally: For control plane load injection (Current PoC we don't use Rally
Shaker: For data plane load injection
os-fault: fault injection ( lot of work need to be done here)

Here is a one example:
Eris is focus on realistic outage scenarios such as, SDN controller
failure, Storage back-end failure, infra L2/L3 SW failure ,etc,,
If those are vendor specific appliances, then how should we do
failure injection and metrics gathering?
Eris will provide plugin mechanism to plug vendor drivers ( provide
by the vendor).
In testing, Eris will use Rally and Shaker for load generation and
inject deterministic/random HW/SW failures in those
appliances and gather the metrics. Then, Eris (or Rally or any other
tool or mix of them) can analyze the result and
create the final report.

Sorry for ranting.....

--- Regards,
Sampath

On Wed, Nov 1, 2017 at 4:02 PM, Boris Pavlovic boris@pavlovic.me wrote:
Sam,

I really don't want to sound like a person that say use Rally my best ever
project blablbab and other BS.
I think that "reinventing wheels" approach is how humanity evolves and
that's why I like this effort in any case.

But really, I read carefully etherpad and I really see in description of
Eris just plain Rally as is:

  • Rally allows you to create tests as YAML
  • Rally allows you to inject in various actions during the load (Rally
    Hooks) which makes it easy to do chaos and other kind of testing
  • Rally is pluggable and you can write even your own Runners (scenario
    executors) that will generate load pattern that you need
  • Rally has SLAs plugins (that can deeply analyze result of test cases) and
    say whatever they pass or not
  • We are working on feature that allows you to mix different workloads in
    parallel (and generate more realistic load)
    -.....

So it would be really nice if you can share gaps that you faced that are
blocking you to use directly Rally..

Thanks!

Best regards,
Boris Pavlovic

On Tue, Oct 31, 2017 at 10:50 PM, Sam P sam47priya@gmail.com wrote:

Hi All,

Sending out a gentle reminder of Sydney Summit Forum Session
regarding this topic.

Extreme/Destructive Testing
Tuesday, November 7, 1:50pm-2:30pm
Sydney Convention and Exhibition Centre - Level 4 - C4.11

[https://www.openstack.org/summit/sydney-2017/summit-schedule/events/20470/extremedestructive-testing]
Eatherpad: https://etherpad.openstack.org/p/SYD-extreme-testing

Your participation in this session would be greatly appreciated.
--- Regards,
Sampath

On Mon, Aug 14, 2017 at 11:43 PM, Tim Bell Tim.Bell@cern.ch wrote:

+1 for Boris’ suggestion. Many of us use Rally to probe our clouds and
have
significant tooling behind it to integrate with local availability
reporting
and trouble ticketing systems. It would be much easier to deploy new
functionality such as you propose if it was integrated into an existing
project framework (such as Rally).

Tim

From: Boris Pavlovic boris@pavlovic.me
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
openstack-dev@lists.openstack.org
Date: Monday, 14 August 2017 at 12:57
To: "OpenStack Development Mailing List (not for usage questions)"
openstack-dev@lists.openstack.org
Cc: openstack-operators openstack-operators@lists.openstack.org
Subject: Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack
Extreme
Testing

Sam,

Seems like a good plan and huge topic ;)

I would as well suggest to take a look at the similar efforts in
OpenStack:

https://rally.readthedocs.io/en/latest/plugins/implementation/hook_and_trigger_plugins.html

Best regards,
Boris Pavlovic

On Mon, Aug 14, 2017 at 2:35 AM, Sam P sam47priya@gmail.com wrote:

Hi All,

This is a follow up for OpenStack Extreme Testing session[1]
we did in MEX-ops-meetup.

Quick intro for those who were not there:
In this work, we proposed to add new testing framework for openstack.
This framework will provides tool for create tests with destructive
scenarios which will check for High Availability, failover and
recovery of OpenStack cloud.
Please refer the link on top of the [1] for further details.

Follow up:
We are planning periodic irc meeting and have an irc
channel for discussion. I will get back to you with those details soon.

At that session, we did not have time to discuss last 3 items,
Reference architectures
We are discussing about the reference architecture in [2].

What sort of failures do you see today in your environment?
Currently we are considering, service failures, backend services (mq,
DB, etc.) failures,
Network sw failures..etc. To begin with the implementation, we are
considering to start with
service failures. Please let us know what failures are more frequent
in your environment.

Emulation/Simulation mechanisms, etc.
Rather than doing actual scale, load, or performance tests, we are
thinking to build a emulation/simulation mechanism
to get the predictions or result of how will openstack behave on such
situations.
This interesting idea was proposed by the Gautam and need more
discussion on this.

Please let us know you questions or comments.

Request to Mike Perez:
We discussed about synergies with openstack assertion tags and other
efforts to do similar testing in openstack.
Could you please give some info or pointer of previous discussions.

[1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
[2]

https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/15477787/Extreme+Testing-Vision+Arch

--- Regards,
Sampath


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Nov 1, 2017 by Sam_P (1,740 points)   3
...