On Tue, 2016-01-12 at 12:52 +0100, Jiri Tomasek wrote:
On 01/11/2016 04:51 PM, Dan Prince wrote:
We've got a problem in TripleO at the moment where many of our
workflows can be driven by the command line only. This causes some
problems for those trying to build a UI around the workflows in
they have to duplicate deployment logic in potentially multiple
There are specs up for review which outline how we might solve this
problem by building what is called TripleO API .
Late last year I began experimenting with an OpenStack service
Mistral which contains a generic workflow API. Mistral supports
defining workflows in YAML and then creating, managing, and
them via an OpenStack API. Initially the effort was focused around
idea of creating a workflow in Mistral which could supplant our
"baremetal introspection" workflow which currently lives in python-
tripleoclient. I create a video presentation which outlines this
. This particular workflow seemed to fit nicely within the
More recently I've turned my attention to what it might look like
were to use Mistral as a replacement for the TripleO API entirely.
brings forth the question of would TripleO be better off building
its own API... or would relying on existing OpenStack APIs be a
Some things I like about the Mistral solution:
The API already exists and is generic.
Mistral already supports interacting with many of the OpenStack
we require . Integration with keystone is baked in. Adding
for new clients seems straightforward (I've had no issues in adding
support for ironic, inspector, and swift actions).
Mistral actions are pluggable. We could fairly easily wrap some
our more complex workflows (perhaps those that aren't easy to
with pure YAML workflows) by creating our own TripleO Mistral
This approach would be similar to creating a custom Heat
something we have avoided with Heat in TripleO but I think it is
perhaps more reasonable with Mistral and would allow us to again
out our YAML workflows to drive things. This might allow us to
off some of the tripleo-common consolidation that is already
We could achieve a "stable API" by simply maintaining input
parameters for workflows in a stable manner. Or perhaps workflows
versioned like a normal API would be as well.
The purist part of me likes Mistral quite a bit. It fits nicely
the deploy OpenStack with OpenStack. I sort of feel like if we have
build our own API in TripleO part of this vision has failed and
even be seen as a massive technical debt which would likely be hard
build a community around outside of TripleO.
Some of the proposed validations could perhaps be implemented as
Mistral actions as well. I'm not convinced we require TripleO API
to support a validations mechanism yet. Perhaps validations seem
because we are simply trying to do them in the wrong places anyway?
(like for example perhaps we should validate network connectivity
inspection time rather than during provisioning).
Power users might find a workflow built around a Mistral API more
easy to interact with and expand upon. Perhaps this ends up being
something that gets submitted as a patchset back to the TripleO
accept into our upstream "stock" workflow sets.
Last week we landed the last patches  to our undercloud to
installing Mistral by simply setting: enable_mistral = true in
undercloud.conf. NOTE: you'll need to be using a recent trunk repo
Delorean so that you have the recently added Mistral packages for
to work. Although the feature is disable by default this should
those wishing to tinker with Mistral as a new TripleO undercloud
service an easy path forwards.
OpenStack Development Mailing List (not for usage questions)
Hi, I have a few questions:
Is Mistral action able to access/manipulate local files? E.g. access
templates installed at undercloud's
Yes. I'm actually working some prototype Mistral actions for tripleo-
common which will do just this to dovetail into a deployment workflow.
(I had to take care of a few heatclient things first though).
I Mistral action able to call either OpenStack service python client
OpenStack service API directly?
The Mistral actions themselves would be written in Python. So I think
the answer here is yes... we could easily make use of existing
OpenStack python clients, or go at the API's ourselves.
What is the response from the Mistral action in the workflow? Lets
we'd use Mistral to get a list of available environments (we do this
tripleo-common now) So we call Mistral API to trigger a workflow
has single action which gets the list of environments. Is mistral
to provide this list as a response, or it is able just to trigger a
Yes, the response from individual actions is quite flexible and can
really be just about anything we want so long at it is serializable I
Probably best to look directly at the Mistral action base class for the
best answer here I think though:
The action result could then be processed by YAQL in a Mistral workflow
and then used in subsequent workflow items or treated as an output to
the workflow itself.
In similar manner, is Mistral able to provide a way to get workflow
progress data? E.g. We have a Mistral workflow for nodes
This workflow contains several actions calling to ironic and
ironic-inspector apis. GUI calls Mistral API to trigger the workflow
now it needs to have a way to track the progress of that workflow.
would this be achieved?
I've been running 'mistral execution-list' and then you can watch
(poll) for the relevant execution items to finish running. Sure,
polling isn't great but I'd say lets start with this perhaps.
I think forcing GUI to poll the APIs that are
called in the actions and try to implement logic that estimates what
state the workflow is to report about it is not a valid solution. We
need the workflow API (Mistral) to provide a lets say web sockets
connection and push the status of actions along with relevant data,
GUI can listen to those.
I don't think Mistral has a websockets implementation. I think Zaqar
does though, and I think perhaps one way we could go about this sort of
notification might be to integrate our workflows with a Zaqar queue or
something. GUI would listen to a Zaqar queue for example... and the
workflow (as it executes) would post things to a specific queue.
Perhaps this is opt-in, only if a Zaqar queue is provided to a given
workflow. FWIW, integrating Mistral w/ Zaqar actions would likely be
Alternately we could look at what it would take to add websocket
support to Mistral directly.