settingsLogin | Registersettings

[openstack-dev] [Fuel] Pluggable framework in Fuel: first prototype ready

0 votes

Hi all,
I moved this conversation to openstack-dev to get a broader audience, since
we started to discuss technical details.

Raw notes from demo session:
https://etherpad.openstack.org/p/cinder-neutron-plugins-second-demo.

Let me start answering on a few questions below from Roman & Nathan.

How are we planning to distribute fuel plugin builder and its updates?
Ideally, it should be available externally (outside of master node). I
don't want us to repeat the same mistake as we did with Fuel client, which
doesn't seem to be usable as an external dependency.

The plan was to have Fuel Plugin Builder (fpb) on PyPI. Ideally it should
be backward compatible with older Fuel release, i.e. when there is Fuel 7.0
out, you should be still able to create plugin for Fuel 6.0. If that it is
going to be overcomplicated - I suggested to produce fpb for every Fuel
release, and name it like fpb60, fpb61, fpb70, etc. Then it becomes easier
to support and maintain plugin builders for certain versions of Fuel.
Speaking about Fuel Client - there is no mistake. It's been discussed
dozens of times, it's just lack of resources to make it on PyPI as well as
to fix a few other things. I hope it could be done as part of efforts from
[2].

  • Perhaps we have a separate settings tab just for Plug-Ins? For some
    > complex plug-ins, they might require a dedicated tab. If we have too many
    > tabs it could get messy.

Shall we consider a separate place in UI (tab) for plugins? Settings tab
seems to be overloaded.

This is certainly under planning and discussion for future releases. See
[1], for example. For 6.0, we agreed that we can just extend existing
Settings tab with plugins-related fields.

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I understand it might sound like a bikeshedding now,
but I believe if we develop it right from the very beginning, then we can
save some time later. So I would suggest normal, short INFO output, and
verbose one with --debug.

Thanks for feedback folks!!!

[1]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37196.html
[2]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37001.html

---------- Forwarded message ----------
From: Nathan Trueblood
Date: Sat, Oct 18, 2014 at 3:24 AM
Subject: Re: plugins

Agreed - I thought this initial PoC was great.

A few initial thoughts about settings in the UI and plug-in in general:

  • Perhaps we have a separate settings tab just for Plug-Ins? For some
    complex plug-ins, they might require a dedicated tab. If we have too many
    tabs it could get messy.
  • It seems like we should consider how we handle the VMWare settings in
    light of plug-ins as well. Since with VMWare we have a lot of setting to
    configure and settings validation.
  • Do we offer any kind of validation for settings on plug-ins? Or some
    way to for the developer to ensure that setting that cannot be default or
    computed get requested for the plug-in?

  • We need to think carefully about both the plug-in developer experience
    (how hard to test, get error messages, etc) and the experience for the user
    who deploys the plug-in into an environment.

-Nathan

On Fri, Oct 17, 2014 at 4:13 PM, Roman Alekseenkov <
ralekseenkov at mirantis.com> wrote:

I watched both videos (creating a file with the text from UI && installing
and starting a service).

It looks pretty good!! Some initial feedback/questions:

  1. I like the fact that fuel plugin builder appends version to the
    name and makes it "fuel-awesome-plugin-1.2.3.tar". The approach is similar
    to Java/Maven and is a good one.
  2. I feel like we should not require user to unpack the plugin before
    installing it. Moreover, we may chose to distribute plugins in our own
    format, which we may potentially change later. E.g. "lbaas-v2.0.fp". I'd
    rather stick with two actions:

    • Assembly (externally): fpb --build
    • Installation (on master node): fuel --install-plugin
  3. How are we planning to distribute fuel plugin builder and its
    updates? Ideally, it should be available externally (outside of master
    node). I don't want us to repeat the same mistake as we did with Fuel
    client, which doesn't seem to be usable as an external dependency.
  4. How do we handle errors?

    • What happens if an error occurs during plugin installation?
    • What happens if an error occurs during plugin execution? Does it
      (should it?) fail the deployment? Will we show user an error message with
      the name of plugin that failed?




    1. Shall we consider a separate place in UI (tab) for plugins?
      Settings tab seems to be overloaded.
  5. When are we planning to focus on the 2 plugins which were
    identified as must-haves for 6.0? Cinder & LBaaS

Once again, great job guys!

Thanks,
Roman

On Fri, Oct 17, 2014 at 9:32 AM, Mike Scherbakov <mscherbakov at mirantis.com

wrote:

Thanks, Evgeny, excellent work!
Roman, I believe we are "green" with the feature. Watch yourself.

Mike Scherbakov

mihgen

On Oct 17, 2014 8:25 PM, "Evgeniy L" wrote:

--
Mike Scherbakov

mihgen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openstack.org/pipermail/openstack-dev/attachments/20141019/a60938d7/attachment.html

asked Oct 19, 2014 in openstack-dev by Mike_Scherbakov (9,460 points)   1 4 6
retagged Jan 28, 2015 by admin

5 Responses

0 votes

Hi guys,

Romans' questions:

I feel like we should not require user to unpack the plugin before
installing it.
Moreover, we may chose to distribute plugins in our own format, which we
may potentially change later. E.g. "lbaas-v2.0.fp".

I like the idea of putting plugin installation functionality in fuel
client, which is installed
on master node.
But in the current version plugin installation requires files operations on
the master,
as result we can have problems if user's fuel-client is installed on
another env.
What we can do is to try to determine where fuel-client is installed, if
it's master
node, we can perform installation, if it isn't master node, we can show
user the
message, that in the current version remote plugin installation is not
supported.
In the next versions if we implement plugin manager (which is separate
service
for plugins management) we will be able to do it remotely.

How are we planning to distribute fuel plugin builder and its updates?

Yes, as Mike mentioned our plan is to release it on PyPi which is python
packages
repository, so any developer will be able to run pip install fpb and get
the tool.

What happens if an error occurs during plugin installation?

Plugins installation process is very simple, our plan is to have some kind
of transaction,
to make it atomic.

  1. register plugin via API
  2. copy the files

In case of error on the 1st step, we can do nothing, in case of error on
the 2nd step,
remove files if there are any, and delete a plugin via rest api. And show
user a message.

What happens if an error occurs during plugin execution?

In the first iteration we are going to interrupt deployment if there are
any errors for plugin's
tasks, also we are thinking how to improve it, for example we wanted to
provide a special
flag for each task, like faildeploumenton_error, and only if it's true,
we fail deployment in
case of failed task. But it can be tricky to implement, it requires to
change the current
orchestrator/nailgun error handling logic. So, I'm not sure if we can
implement this logic in
the first release.

Regarding to meaningful error messages, yes, we want to show the
user, which plugin
causes the error.

Shall we consider a separate place in UI (tab) for plugins?

+1 to Mike's answer

When are we planning to focus on the 2 plugins which were identified as
must-haves
for 6.0? Cinder & LBaaS

For Cinder we are going to implement plugin which configures GlusterFS as
cinder backend,
so, if user has installed GlusterFS cluster, we can configure our cinder to
work with it,
I want to mention that we don't install GlusterFS nodes, we just configure
cinder to work
with user's GlusterFS cluster.
Stanislaw B. already did some scripts which configures cinder to work with
GlusterFS, so
we are on testing stage.

Regarding to LBaaS, Stanislaw B. did multinode implementation, ha
implementation is tricky
and requires some additional work, we are working on it.

Nathan's questions:

Looks like Mike answered UI related questions.

Do we offer any kind of validation for settings on plug-ins? Or some
way to for the developer
to ensure that setting that cannot be default or computed get requested
for the plug-in?

Yes, each field can have regexp which is used during the validation.

Mike's questions:

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I
understand it might sound like a bikeshedding now, but I believe if we
develop it right from
the very beginning, then we can save some time later. So I would suggest
normal, short INFO
output, and verbose one with --debug.

Agree.

Thanks for your feedback,

On Sun, Oct 19, 2014 at 1:11 PM, Mike Scherbakov
wrote:

Hi all,
I moved this conversation to openstack-dev to get a broader audience,
since we started to discuss technical details.

Raw notes from demo session:
https://etherpad.openstack.org/p/cinder-neutron-plugins-second-demo.

Let me start answering on a few questions below from Roman & Nathan.

How are we planning to distribute fuel plugin builder and its updates?
Ideally, it should be available externally (outside of master node). I
don't want us to repeat the same mistake as we did with Fuel client, which
doesn't seem to be usable as an external dependency.

The plan was to have Fuel Plugin Builder (fpb) on PyPI. Ideally it should
be backward compatible with older Fuel release, i.e. when there is Fuel 7.0
out, you should be still able to create plugin for Fuel 6.0. If that it is
going to be overcomplicated - I suggested to produce fpb for every Fuel
release, and name it like fpb60, fpb61, fpb70, etc. Then it becomes easier
to support and maintain plugin builders for certain versions of Fuel.
Speaking about Fuel Client - there is no mistake. It's been discussed
dozens of times, it's just lack of resources to make it on PyPI as well as
to fix a few other things. I hope it could be done as part of efforts from
[2].

  • Perhaps we have a separate settings tab just for Plug-Ins? For some
    > complex plug-ins, they might require a dedicated tab. If we have too many
    > tabs it could get messy.

Shall we consider a separate place in UI (tab) for plugins? Settings tab
seems to be overloaded.

This is certainly under planning and discussion for future releases. See
[1], for example. For 6.0, we agreed that we can just extend existing
Settings tab with plugins-related fields.

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I understand it might sound like a bikeshedding now,
but I believe if we develop it right from the very beginning, then we can
save some time later. So I would suggest normal, short INFO output, and
verbose one with --debug.

Thanks for feedback folks!!!

[1]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37196.html
[2]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37001.html

---------- Forwarded message ----------
From: Nathan Trueblood
Date: Sat, Oct 18, 2014 at 3:24 AM
Subject: Re: plugins

Agreed - I thought this initial PoC was great.

A few initial thoughts about settings in the UI and plug-in in general:

  • Perhaps we have a separate settings tab just for Plug-Ins? For some
    complex plug-ins, they might require a dedicated tab. If we have too many
    tabs it could get messy.
  • It seems like we should consider how we handle the VMWare settings in
    light of plug-ins as well. Since with VMWare we have a lot of setting to
    configure and settings validation.
  • Do we offer any kind of validation for settings on plug-ins? Or some
    way to for the developer to ensure that setting that cannot be default or
    computed get requested for the plug-in?

  • We need to think carefully about both the plug-in developer experience
    (how hard to test, get error messages, etc) and the experience for the user
    who deploys the plug-in into an environment.

-Nathan

On Fri, Oct 17, 2014 at 4:13 PM, Roman Alekseenkov <
ralekseenkov at mirantis.com> wrote:

I watched both videos (creating a file with the text from UI &&
installing and starting a service).

It looks pretty good!! Some initial feedback/questions:

  1. I like the fact that fuel plugin builder appends version to the
    name and makes it "fuel-awesome-plugin-1.2.3.tar". The approach is similar
    to Java/Maven and is a good one.
  2. I feel like we should not require user to unpack the plugin before
    installing it. Moreover, we may chose to distribute plugins in our own
    format, which we may potentially change later. E.g. "lbaas-v2.0.fp". I'd
    rather stick with two actions:

    • Assembly (externally): fpb --build
    • Installation (on master node): fuel --install-plugin
  3. How are we planning to distribute fuel plugin builder and its
    updates? Ideally, it should be available externally (outside of master
    node). I don't want us to repeat the same mistake as we did with Fuel
    client, which doesn't seem to be usable as an external dependency.
  4. How do we handle errors?

    • What happens if an error occurs during plugin installation?
    • What happens if an error occurs during plugin execution? Does it
      (should it?) fail the deployment? Will we show user an error message with
      the name of plugin that failed?




    1. Shall we consider a separate place in UI (tab) for plugins?
      Settings tab seems to be overloaded.
  5. When are we planning to focus on the 2 plugins which were
    identified as must-haves for 6.0? Cinder & LBaaS

Once again, great job guys!

Thanks,
Roman

On Fri, Oct 17, 2014 at 9:32 AM, Mike Scherbakov <
mscherbakov at mirantis.com> wrote:

Thanks, Evgeny, excellent work!
Roman, I believe we are "green" with the feature. Watch yourself.

Mike Scherbakov

mihgen

On Oct 17, 2014 8:25 PM, "Evgeniy L" wrote:

--
Mike Scherbakov

mihgen


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Oct 20, 2014 by Evgeniy_L (8,580 points)   1 2 4
0 votes

Hi,

As for a separate section for plugins, I think we should not force it and
leave this decision to a plugin developer, so he can create just a single
checkbox or a section of the settings tab or a separate tab depending on
plugin functionality. Plugins should be able to modify arbitrary release
fields. For example, if Ceph was a plugin, it should be able to extend
wizard config to add new options to Storage pane. If vCenter was a plugin,
it should be able to set maximum amount of Compute nodes to 0.

2014-10-20 21:21 GMT+07:00 Evgeniy L :

Hi guys,

Romans' questions:

I feel like we should not require user to unpack the plugin before
installing it.
Moreover, we may chose to distribute plugins in our own format, which we
may potentially change later. E.g. "lbaas-v2.0.fp".

I like the idea of putting plugin installation functionality in fuel
client, which is installed
on master node.
But in the current version plugin installation requires files operations
on the master,
as result we can have problems if user's fuel-client is installed on
another env.
What we can do is to try to determine where fuel-client is installed, if
it's master
node, we can perform installation, if it isn't master node, we can show
user the
message, that in the current version remote plugin installation is not
supported.
In the next versions if we implement plugin manager (which is separate
service
for plugins management) we will be able to do it remotely.

How are we planning to distribute fuel plugin builder and its updates?

Yes, as Mike mentioned our plan is to release it on PyPi which is python
packages
repository, so any developer will be able to run pip install fpb and get
the tool.

What happens if an error occurs during plugin installation?

Plugins installation process is very simple, our plan is to have some kind
of transaction,
to make it atomic.

  1. register plugin via API
  2. copy the files

In case of error on the 1st step, we can do nothing, in case of error on
the 2nd step,
remove files if there are any, and delete a plugin via rest api. And show
user a message.

What happens if an error occurs during plugin execution?

In the first iteration we are going to interrupt deployment if there are
any errors for plugin's
tasks, also we are thinking how to improve it, for example we wanted to
provide a special
flag for each task, like faildeploumenton_error, and only if it's true,
we fail deployment in
case of failed task. But it can be tricky to implement, it requires to
change the current
orchestrator/nailgun error handling logic. So, I'm not sure if we can
implement this logic in
the first release.

Regarding to meaningful error messages, yes, we want to show the
user, which plugin
causes the error.

Shall we consider a separate place in UI (tab) for plugins?

+1 to Mike's answer

When are we planning to focus on the 2 plugins which were identified
as must-haves
for 6.0? Cinder & LBaaS

For Cinder we are going to implement plugin which configures GlusterFS as
cinder backend,
so, if user has installed GlusterFS cluster, we can configure our cinder
to work with it,
I want to mention that we don't install GlusterFS nodes, we just configure
cinder to work
with user's GlusterFS cluster.
Stanislaw B. already did some scripts which configures cinder to work with
GlusterFS, so
we are on testing stage.

Regarding to LBaaS, Stanislaw B. did multinode implementation, ha
implementation is tricky
and requires some additional work, we are working on it.

Nathan's questions:

Looks like Mike answered UI related questions.

Do we offer any kind of validation for settings on plug-ins? Or some
way to for the developer
to ensure that setting that cannot be default or computed get requested
for the plug-in?

Yes, each field can have regexp which is used during the validation.

Mike's questions:

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I
understand it might sound like a bikeshedding now, but I believe if we
develop it right from
the very beginning, then we can save some time later. So I would
suggest normal, short INFO
output, and verbose one with --debug.

Agree.

Thanks for your feedback,

On Sun, Oct 19, 2014 at 1:11 PM, Mike Scherbakov <mscherbakov at mirantis.com

wrote:

Hi all,
I moved this conversation to openstack-dev to get a broader audience,
since we started to discuss technical details.

Raw notes from demo session:
https://etherpad.openstack.org/p/cinder-neutron-plugins-second-demo.

Let me start answering on a few questions below from Roman & Nathan.

How are we planning to distribute fuel plugin builder and its updates?
Ideally, it should be available externally (outside of master node). I
don't want us to repeat the same mistake as we did with Fuel client, which
doesn't seem to be usable as an external dependency.

The plan was to have Fuel Plugin Builder (fpb) on PyPI. Ideally it should
be backward compatible with older Fuel release, i.e. when there is Fuel 7.0
out, you should be still able to create plugin for Fuel 6.0. If that it is
going to be overcomplicated - I suggested to produce fpb for every Fuel
release, and name it like fpb60, fpb61, fpb70, etc. Then it becomes easier
to support and maintain plugin builders for certain versions of Fuel.
Speaking about Fuel Client - there is no mistake. It's been discussed
dozens of times, it's just lack of resources to make it on PyPI as well as
to fix a few other things. I hope it could be done as part of efforts from
[2].

  • Perhaps we have a separate settings tab just for Plug-Ins? For some
    > complex plug-ins, they might require a dedicated tab. If we have too many
    > tabs it could get messy.

Shall we consider a separate place in UI (tab) for plugins? Settings tab
seems to be overloaded.

This is certainly under planning and discussion for future releases. See
[1], for example. For 6.0, we agreed that we can just extend existing
Settings tab with plugins-related fields.

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I understand it might sound like a bikeshedding now,
but I believe if we develop it right from the very beginning, then we can
save some time later. So I would suggest normal, short INFO output, and
verbose one with --debug.

Thanks for feedback folks!!!

[1]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37196.html
[2]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37001.html

---------- Forwarded message ----------
From: Nathan Trueblood
Date: Sat, Oct 18, 2014 at 3:24 AM
Subject: Re: plugins

Agreed - I thought this initial PoC was great.

A few initial thoughts about settings in the UI and plug-in in general:

  • Perhaps we have a separate settings tab just for Plug-Ins? For some
    complex plug-ins, they might require a dedicated tab. If we have too many
    tabs it could get messy.
  • It seems like we should consider how we handle the VMWare settings in
    light of plug-ins as well. Since with VMWare we have a lot of setting to
    configure and settings validation.
  • Do we offer any kind of validation for settings on plug-ins? Or some
    way to for the developer to ensure that setting that cannot be default or
    computed get requested for the plug-in?

  • We need to think carefully about both the plug-in developer experience
    (how hard to test, get error messages, etc) and the experience for the user
    who deploys the plug-in into an environment.

-Nathan

On Fri, Oct 17, 2014 at 4:13 PM, Roman Alekseenkov <
ralekseenkov at mirantis.com> wrote:

I watched both videos (creating a file with the text from UI &&
installing and starting a service).

It looks pretty good!! Some initial feedback/questions:

  1. I like the fact that fuel plugin builder appends version to the
    name and makes it "fuel-awesome-plugin-1.2.3.tar". The approach is similar
    to Java/Maven and is a good one.
  2. I feel like we should not require user to unpack the plugin
    before installing it. Moreover, we may chose to distribute plugins in our
    own format, which we may potentially change later. E.g. "lbaas-v2.0.fp".
    I'd rather stick with two actions:

    • Assembly (externally): fpb --build
    • Installation (on master node): fuel --install-plugin
  3. How are we planning to distribute fuel plugin builder and its
    updates? Ideally, it should be available externally (outside of master
    node). I don't want us to repeat the same mistake as we did with Fuel
    client, which doesn't seem to be usable as an external dependency.
  4. How do we handle errors?

    • What happens if an error occurs during plugin installation?
    • What happens if an error occurs during plugin execution? Does
      it (should it?) fail the deployment? Will we show user an error message
      with the name of plugin that failed?




    1. Shall we consider a separate place in UI (tab) for plugins?
      Settings tab seems to be overloaded.
  5. When are we planning to focus on the 2 plugins which were
    identified as must-haves for 6.0? Cinder & LBaaS

Once again, great job guys!

Thanks,
Roman

On Fri, Oct 17, 2014 at 9:32 AM, Mike Scherbakov <
mscherbakov at mirantis.com> wrote:

Thanks, Evgeny, excellent work!
Roman, I believe we are "green" with the feature. Watch yourself.

Mike Scherbakov

mihgen

On Oct 17, 2014 8:25 PM, "Evgeniy L" wrote:

--
Mike Scherbakov

mihgen


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Vitaly Kramskikh,
Software Engineer,
Mirantis, Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Oct 21, 2014 by Vitaly_Kramskikh (3,480 points)   2 4
0 votes
  1. I feel like we should not require user to unpack the plugin before
    installing it. Moreover, we may chose to distribute plugins in our own
    format, which we may potentially change later. E.g. "lbaas-v2.0.fp". I'd
    rather stick with two actions:
  • Assembly (externally): fpb --build

  • Installation (on master node): fuel --install-plugin

    I like the idea of putting plugin installation functionality in fuel client,
    which is installed
    on master node.
    But in the current version plugin installation requires files operations
    on the master,
    as result we can have problems if user's fuel-client is installed on
    another env.

I suggest to keep it simple for now as we have the issue mentioned by
Evgeny: fuel client is supposed to work from other nodes, and we will need
additional verification code in there. Also, to make it smooth, we will
have to end up with a few more checks - like what if tarball is broken,
what if we can't find install script in it, etc.
I'd suggest to run it simple for 6.0, and then we will see how it's being
used and what other limitations / issues we have around plugin installation
and usage. We can consider to make this functionality as part of fuel
client a bit later.

Thanks,

On Tue, Oct 21, 2014 at 6:57 PM, Vitaly Kramskikh
wrote:

Hi,

As for a separate section for plugins, I think we should not force it and
leave this decision to a plugin developer, so he can create just a single
checkbox or a section of the settings tab or a separate tab depending on
plugin functionality. Plugins should be able to modify arbitrary release
fields. For example, if Ceph was a plugin, it should be able to extend
wizard config to add new options to Storage pane. If vCenter was a plugin,
it should be able to set maximum amount of Compute nodes to 0.

2014-10-20 21:21 GMT+07:00 Evgeniy L :

Hi guys,

Romans' questions:

I feel like we should not require user to unpack the plugin before
installing it.
Moreover, we may chose to distribute plugins in our own format, which
we
may potentially change later. E.g. "lbaas-v2.0.fp".

I like the idea of putting plugin installation functionality in fuel
client, which is installed
on master node.
But in the current version plugin installation requires files operations
on the master,
as result we can have problems if user's fuel-client is installed on
another env.
What we can do is to try to determine where fuel-client is installed, if
it's master
node, we can perform installation, if it isn't master node, we can show
user the
message, that in the current version remote plugin installation is not
supported.
In the next versions if we implement plugin manager (which is separate
service
for plugins management) we will be able to do it remotely.

How are we planning to distribute fuel plugin builder and its updates?

Yes, as Mike mentioned our plan is to release it on PyPi which is python
packages
repository, so any developer will be able to run pip install fpb and
get the tool.

What happens if an error occurs during plugin installation?

Plugins installation process is very simple, our plan is to have some
kind of transaction,
to make it atomic.

  1. register plugin via API
  2. copy the files

In case of error on the 1st step, we can do nothing, in case of error on
the 2nd step,
remove files if there are any, and delete a plugin via rest api. And show
user a message.

What happens if an error occurs during plugin execution?

In the first iteration we are going to interrupt deployment if there are
any errors for plugin's
tasks, also we are thinking how to improve it, for example we wanted to
provide a special
flag for each task, like faildeploumenton_error, and only if it's true,
we fail deployment in
case of failed task. But it can be tricky to implement, it requires to
change the current
orchestrator/nailgun error handling logic. So, I'm not sure if we can
implement this logic in
the first release.

Regarding to meaningful error messages, yes, we want to show the
user, which plugin
causes the error.

Shall we consider a separate place in UI (tab) for plugins?

+1 to Mike's answer

When are we planning to focus on the 2 plugins which were identified
as must-haves
for 6.0? Cinder & LBaaS

For Cinder we are going to implement plugin which configures GlusterFS as
cinder backend,
so, if user has installed GlusterFS cluster, we can configure our cinder
to work with it,
I want to mention that we don't install GlusterFS nodes, we just
configure cinder to work
with user's GlusterFS cluster.
Stanislaw B. already did some scripts which configures cinder to work
with GlusterFS, so
we are on testing stage.

Regarding to LBaaS, Stanislaw B. did multinode implementation, ha
implementation is tricky
and requires some additional work, we are working on it.

Nathan's questions:

Looks like Mike answered UI related questions.

Do we offer any kind of validation for settings on plug-ins? Or some
way to for the developer
to ensure that setting that cannot be default or computed get
requested for the plug-in?

Yes, each field can have regexp which is used during the validation.

Mike's questions:

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I
understand it might sound like a bikeshedding now, but I believe if we
develop it right from
the very beginning, then we can save some time later. So I would
suggest normal, short INFO
output, and verbose one with --debug.

Agree.

Thanks for your feedback,

On Sun, Oct 19, 2014 at 1:11 PM, Mike Scherbakov <
mscherbakov at mirantis.com> wrote:

Hi all,
I moved this conversation to openstack-dev to get a broader audience,
since we started to discuss technical details.

Raw notes from demo session:
https://etherpad.openstack.org/p/cinder-neutron-plugins-second-demo.

Let me start answering on a few questions below from Roman & Nathan.

How are we planning to distribute fuel plugin builder and its updates?
Ideally, it should be available externally (outside of master node). I
don't want us to repeat the same mistake as we did with Fuel client, which
doesn't seem to be usable as an external dependency.

The plan was to have Fuel Plugin Builder (fpb) on PyPI. Ideally it
should be backward compatible with older Fuel release, i.e. when there is
Fuel 7.0 out, you should be still able to create plugin for Fuel 6.0. If
that it is going to be overcomplicated - I suggested to produce fpb for
every Fuel release, and name it like fpb60, fpb61, fpb70, etc. Then it
becomes easier to support and maintain plugin builders for certain versions
of Fuel.
Speaking about Fuel Client - there is no mistake. It's been discussed
dozens of times, it's just lack of resources to make it on PyPI as well as
to fix a few other things. I hope it could be done as part of efforts from
[2].

  • Perhaps we have a separate settings tab just for Plug-Ins? For some
    > complex plug-ins, they might require a dedicated tab. If we have too many
    > tabs it could get messy.

Shall we consider a separate place in UI (tab) for plugins? Settings
tab seems to be overloaded.

This is certainly under planning and discussion for future releases. See
[1], for example. For 6.0, we agreed that we can just extend existing
Settings tab with plugins-related fields.

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I understand it might sound like a bikeshedding now,
but I believe if we develop it right from the very beginning, then we can
save some time later. So I would suggest normal, short INFO output, and
verbose one with --debug.

Thanks for feedback folks!!!

[1]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37196.html
[2]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37001.html

---------- Forwarded message ----------
From: Nathan Trueblood
Date: Sat, Oct 18, 2014 at 3:24 AM
Subject: Re: plugins

Agreed - I thought this initial PoC was great.

A few initial thoughts about settings in the UI and plug-in in general:

  • Perhaps we have a separate settings tab just for Plug-Ins? For some
    complex plug-ins, they might require a dedicated tab. If we have too many
    tabs it could get messy.
  • It seems like we should consider how we handle the VMWare settings in
    light of plug-ins as well. Since with VMWare we have a lot of setting to
    configure and settings validation.
  • Do we offer any kind of validation for settings on plug-ins? Or some
    way to for the developer to ensure that setting that cannot be default or
    computed get requested for the plug-in?

  • We need to think carefully about both the plug-in developer experience
    (how hard to test, get error messages, etc) and the experience for the user
    who deploys the plug-in into an environment.

-Nathan

On Fri, Oct 17, 2014 at 4:13 PM, Roman Alekseenkov <
ralekseenkov at mirantis.com> wrote:

I watched both videos (creating a file with the text from UI &&
installing and starting a service).

It looks pretty good!! Some initial feedback/questions:

  1. I like the fact that fuel plugin builder appends version to the
    name and makes it "fuel-awesome-plugin-1.2.3.tar". The approach is similar
    to Java/Maven and is a good one.
  2. I feel like we should not require user to unpack the plugin
    before installing it. Moreover, we may chose to distribute plugins in our
    own format, which we may potentially change later. E.g. "lbaas-v2.0.fp".
    I'd rather stick with two actions:

    • Assembly (externally): fpb --build
    • Installation (on master node): fuel --install-plugin
  3. How are we planning to distribute fuel plugin builder and its
    updates? Ideally, it should be available externally (outside of master
    node). I don't want us to repeat the same mistake as we did with Fuel
    client, which doesn't seem to be usable as an external dependency.
  4. How do we handle errors?

    • What happens if an error occurs during plugin installation?
    • What happens if an error occurs during plugin execution? Does
      it (should it?) fail the deployment? Will we show user an error message
      with the name of plugin that failed?




    1. Shall we consider a separate place in UI (tab) for plugins?
      Settings tab seems to be overloaded.
  5. When are we planning to focus on the 2 plugins which were
    identified as must-haves for 6.0? Cinder & LBaaS

Once again, great job guys!

Thanks,
Roman

On Fri, Oct 17, 2014 at 9:32 AM, Mike Scherbakov <
mscherbakov at mirantis.com> wrote:

Thanks, Evgeny, excellent work!
Roman, I believe we are "green" with the feature. Watch yourself.

Mike Scherbakov

mihgen

On Oct 17, 2014 8:25 PM, "Evgeniy L" wrote:

--
Mike Scherbakov

mihgen


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Vitaly Kramskikh,
Software Engineer,
Mirantis, Inc.


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Mike Scherbakov

mihgen

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Oct 23, 2014 by Mike_Scherbakov (9,460 points)   1 4 6
0 votes

Hi Mike,

I would like to add a bit more details about current implementation and how
it can be done.

Implement installation as a scripts inside of tar ball:
Cons:
* install script is really simple right now, but it will be much more
complicated
** it requires to implement logic where we can ask user for login/password
** use some config, where we will be able to get endpoints, like where is
keystone, nailgun
** validate that it's possible to install plugin on the current version of
master
** handle error cases (to make installation process more atomic)
* it will be impossible to deprecate the installation logic/method, because
it's on the plugin's side
and you cannot change a plugin which user downloaded some times ago, when
we get
plugin manager, we probably would like user to use plugin manager,
instead of some scripts
* plugin installation process is not so simple as it could be (untar, cd
plugin, ./install)

Pros:
* plugin developer can change installation scripts (I'm not sure if it's a
pros)

Add installation to fuel client:
Cons:
* requires changes in fuel client, which are not good for fuel client by
design (fuel client
should be able to work remotely from user's machine), current
implementation requires
local operations on files, it will be changed in the future releases, so
fuel-client will
be able to do it via api, also we can determine if it's not master node
by /etc/fuel/version.yaml
and show the user a message which says that in the current version it's
not possible
to install the plugin remotely
* plugin developer won't be able to change installation process (I'm not
sure if it's a cons)

Pros:
* it's easier for user to install the plugin `fuel --install-plugin
plugin_name-1.0.1.fpb'
* all of the authentication logic already implemented in fuel client
* fuel client uses config with endpoints which is generated by puppet
* it will be easier to deprecate previous installation approach, we can
just install new
fuel client on the master which uses api

Personally I like the second approach, and I think we should try to
implement it,
when we get time.

Thanks,

On Thu, Oct 23, 2014 at 3:02 PM, Mike Scherbakov
wrote:

  1. I feel like we should not require user to unpack the plugin before
    installing it. Moreover, we may chose to distribute plugins in our own
    format, which we may potentially change later. E.g. "lbaas-v2.0.fp". I'd
    rather stick with two actions:
  • Assembly (externally): fpb --build

  • Installation (on master node): fuel --install-plugin

    I like the idea of putting plugin installation functionality in fuel client,
    which is installed
    on master node.
    But in the current version plugin installation requires files operations
    on the master,
    as result we can have problems if user's fuel-client is installed on
    another env.

I suggest to keep it simple for now as we have the issue mentioned by
Evgeny: fuel client is supposed to work from other nodes, and we will need
additional verification code in there. Also, to make it smooth, we will
have to end up with a few more checks - like what if tarball is broken,
what if we can't find install script in it, etc.
I'd suggest to run it simple for 6.0, and then we will see how it's being
used and what other limitations / issues we have around plugin installation
and usage. We can consider to make this functionality as part of fuel
client a bit later.

Thanks,

On Tue, Oct 21, 2014 at 6:57 PM, Vitaly Kramskikh <vkramskikh at mirantis.com

wrote:

Hi,

As for a separate section for plugins, I think we should not force it and
leave this decision to a plugin developer, so he can create just a single
checkbox or a section of the settings tab or a separate tab depending on
plugin functionality. Plugins should be able to modify arbitrary release
fields. For example, if Ceph was a plugin, it should be able to extend
wizard config to add new options to Storage pane. If vCenter was a plugin,
it should be able to set maximum amount of Compute nodes to 0.

2014-10-20 21:21 GMT+07:00 Evgeniy L :

Hi guys,

Romans' questions:

I feel like we should not require user to unpack the plugin before
installing it.
Moreover, we may chose to distribute plugins in our own format, which
we
may potentially change later. E.g. "lbaas-v2.0.fp".

I like the idea of putting plugin installation functionality in fuel
client, which is installed
on master node.
But in the current version plugin installation requires files operations
on the master,
as result we can have problems if user's fuel-client is installed on
another env.
What we can do is to try to determine where fuel-client is installed, if
it's master
node, we can perform installation, if it isn't master node, we can show
user the
message, that in the current version remote plugin installation is not
supported.
In the next versions if we implement plugin manager (which is separate
service
for plugins management) we will be able to do it remotely.

How are we planning to distribute fuel plugin builder and its
updates?

Yes, as Mike mentioned our plan is to release it on PyPi which is python
packages
repository, so any developer will be able to run pip install fpb and
get the tool.

What happens if an error occurs during plugin installation?

Plugins installation process is very simple, our plan is to have some
kind of transaction,
to make it atomic.

  1. register plugin via API
  2. copy the files

In case of error on the 1st step, we can do nothing, in case of error on
the 2nd step,
remove files if there are any, and delete a plugin via rest api. And
show user a message.

What happens if an error occurs during plugin execution?

In the first iteration we are going to interrupt deployment if there are
any errors for plugin's
tasks, also we are thinking how to improve it, for example we wanted to
provide a special
flag for each task, like faildeploumenton_error, and only if it's
true, we fail deployment in
case of failed task. But it can be tricky to implement, it requires to
change the current
orchestrator/nailgun error handling logic. So, I'm not sure if we can
implement this logic in
the first release.

Regarding to meaningful error messages, yes, we want to show the
user, which plugin
causes the error.

Shall we consider a separate place in UI (tab) for plugins?

+1 to Mike's answer

When are we planning to focus on the 2 plugins which were identified
as must-haves
for 6.0? Cinder & LBaaS

For Cinder we are going to implement plugin which configures GlusterFS
as cinder backend,
so, if user has installed GlusterFS cluster, we can configure our cinder
to work with it,
I want to mention that we don't install GlusterFS nodes, we just
configure cinder to work
with user's GlusterFS cluster.
Stanislaw B. already did some scripts which configures cinder to work
with GlusterFS, so
we are on testing stage.

Regarding to LBaaS, Stanislaw B. did multinode implementation, ha
implementation is tricky
and requires some additional work, we are working on it.

Nathan's questions:

Looks like Mike answered UI related questions.

Do we offer any kind of validation for settings on plug-ins? Or
some way to for the developer
to ensure that setting that cannot be default or computed get
requested for the plug-in?

Yes, each field can have regexp which is used during the validation.

Mike's questions:

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I
understand it might sound like a bikeshedding now, but I believe if
we develop it right from
the very beginning, then we can save some time later. So I would
suggest normal, short INFO
output, and verbose one with --debug.

Agree.

Thanks for your feedback,

On Sun, Oct 19, 2014 at 1:11 PM, Mike Scherbakov <
mscherbakov at mirantis.com> wrote:

Hi all,
I moved this conversation to openstack-dev to get a broader audience,
since we started to discuss technical details.

Raw notes from demo session:
https://etherpad.openstack.org/p/cinder-neutron-plugins-second-demo.

Let me start answering on a few questions below from Roman & Nathan.

How are we planning to distribute fuel plugin builder and its updates?
Ideally, it should be available externally (outside of master node). I
don't want us to repeat the same mistake as we did with Fuel client, which
doesn't seem to be usable as an external dependency.

The plan was to have Fuel Plugin Builder (fpb) on PyPI. Ideally it
should be backward compatible with older Fuel release, i.e. when there is
Fuel 7.0 out, you should be still able to create plugin for Fuel 6.0. If
that it is going to be overcomplicated - I suggested to produce fpb for
every Fuel release, and name it like fpb60, fpb61, fpb70, etc. Then it
becomes easier to support and maintain plugin builders for certain versions
of Fuel.
Speaking about Fuel Client - there is no mistake. It's been discussed
dozens of times, it's just lack of resources to make it on PyPI as well as
to fix a few other things. I hope it could be done as part of efforts from
[2].

  • Perhaps we have a separate settings tab just for Plug-Ins? For
    > some complex plug-ins, they might require a dedicated tab. If we have too
    > many tabs it could get messy.

Shall we consider a separate place in UI (tab) for plugins? Settings
tab seems to be overloaded.

This is certainly under planning and discussion for future releases.
See [1], for example. For 6.0, we agreed that we can just extend existing
Settings tab with plugins-related fields.

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I understand it might sound like a bikeshedding now,
but I believe if we develop it right from the very beginning, then we can
save some time later. So I would suggest normal, short INFO output, and
verbose one with --debug.

Thanks for feedback folks!!!

[1]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37196.html
[2]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37001.html

---------- Forwarded message ----------
From: Nathan Trueblood
Date: Sat, Oct 18, 2014 at 3:24 AM
Subject: Re: plugins

Agreed - I thought this initial PoC was great.

A few initial thoughts about settings in the UI and plug-in in general:

  • Perhaps we have a separate settings tab just for Plug-Ins? For
    some complex plug-ins, they might require a dedicated tab. If we have too
    many tabs it could get messy.
  • It seems like we should consider how we handle the VMWare settings in
    light of plug-ins as well. Since with VMWare we have a lot of setting to
    configure and settings validation.
  • Do we offer any kind of validation for settings on plug-ins? Or
    some way to for the developer to ensure that setting that cannot be default
    or computed get requested for the plug-in?

  • We need to think carefully about both the plug-in developer
    experience (how hard to test, get error messages, etc) and the experience
    for the user who deploys the plug-in into an environment.

-Nathan

On Fri, Oct 17, 2014 at 4:13 PM, Roman Alekseenkov <
ralekseenkov at mirantis.com> wrote:

I watched both videos (creating a file with the text from UI &&
installing and starting a service).

It looks pretty good!! Some initial feedback/questions:

  1. I like the fact that fuel plugin builder appends version to the
    name and makes it "fuel-awesome-plugin-1.2.3.tar". The approach is similar
    to Java/Maven and is a good one.
  2. I feel like we should not require user to unpack the plugin
    before installing it. Moreover, we may chose to distribute plugins in our
    own format, which we may potentially change later. E.g. "lbaas-v2.0.fp".
    I'd rather stick with two actions:

    • Assembly (externally): fpb --build
    • Installation (on master node): fuel --install-plugin
  3. How are we planning to distribute fuel plugin builder and its
    updates? Ideally, it should be available externally (outside of master
    node). I don't want us to repeat the same mistake as we did with Fuel
    client, which doesn't seem to be usable as an external dependency.
  4. How do we handle errors?

    • What happens if an error occurs during plugin installation?
    • What happens if an error occurs during plugin execution? Does
      it (should it?) fail the deployment? Will we show user an error message
      with the name of plugin that failed?




    1. Shall we consider a separate place in UI (tab) for plugins?
      Settings tab seems to be overloaded.
  5. When are we planning to focus on the 2 plugins which were
    identified as must-haves for 6.0? Cinder & LBaaS

Once again, great job guys!

Thanks,
Roman

On Fri, Oct 17, 2014 at 9:32 AM, Mike Scherbakov <
mscherbakov at mirantis.com> wrote:

Thanks, Evgeny, excellent work!
Roman, I believe we are "green" with the feature. Watch yourself.

Mike Scherbakov

mihgen

On Oct 17, 2014 8:25 PM, "Evgeniy L" wrote:

--
Mike Scherbakov

mihgen


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Vitaly Kramskikh,
Software Engineer,
Mirantis, Inc.


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Mike Scherbakov

mihgen


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Oct 23, 2014 by Evgeniy_L (8,580 points)   1 2 4
0 votes

Preventing plugin developers from implementing their own installer is
a pro, not a con, you've already listed one reason in cons against
install scripts inside plugin tarball: if we centralize plugin
installation and management logic in fuel, we can change it once for
all plugins and don't have to worry about old plugins using an
obsolete installer.

I think priorities here should be 1) easy of plugin development; and
2) ease of use. Pluggable architecture won't do us much good if we end
up being the only ones being able to use it efficiently. Adding a
little more complexity to fuelclient to allow moving a lot of fuel
complexity from core to plugins is a good tradeoff.

On Thu, Oct 23, 2014 at 8:32 AM, Evgeniy L wrote:
Hi Mike,

I would like to add a bit more details about current implementation and how
it can be done.

Implement installation as a scripts inside of tar ball:
Cons:
* install script is really simple right now, but it will be much more
complicated
** it requires to implement logic where we can ask user for login/password
** use some config, where we will be able to get endpoints, like where is
keystone, nailgun
** validate that it's possible to install plugin on the current version of
master
** handle error cases (to make installation process more atomic)
* it will be impossible to deprecate the installation logic/method, because
it's on the plugin's side
and you cannot change a plugin which user downloaded some times ago, when
we get
plugin manager, we probably would like user to use plugin manager, instead
of some scripts
* plugin installation process is not so simple as it could be (untar, cd
plugin, ./install)

Pros:
* plugin developer can change installation scripts (I'm not sure if it's a
pros)

Add installation to fuel client:
Cons:
* requires changes in fuel client, which are not good for fuel client by
design (fuel client
should be able to work remotely from user's machine), current
implementation requires
local operations on files, it will be changed in the future releases, so
fuel-client will
be able to do it via api, also we can determine if it's not master node by
/etc/fuel/version.yaml
and show the user a message which says that in the current version it's
not possible
to install the plugin remotely
* plugin developer won't be able to change installation process (I'm not
sure if it's a cons)

Pros:
* it's easier for user to install the plugin `fuel --install-plugin
plugin_name-1.0.1.fpb'
* all of the authentication logic already implemented in fuel client
* fuel client uses config with endpoints which is generated by puppet
* it will be easier to deprecate previous installation approach, we can just
install new
fuel client on the master which uses api

Personally I like the second approach, and I think we should try to
implement it,
when we get time.

Thanks,

On Thu, Oct 23, 2014 at 3:02 PM, Mike Scherbakov
wrote:

I feel like we should not require user to unpack the plugin before
installing it. Moreover, we may chose to distribute plugins in our own
format, which we may potentially change later. E.g. "lbaas-v2.0.fp". I'd
rather stick with two actions:

Assembly (externally): fpb --build

Installation (on master node): fuel --install-plugin

I like the idea of putting plugin installation functionality in fuel
client, which is installed
on master node.
But in the current version plugin installation requires files operations
on the master,
as result we can have problems if user's fuel-client is installed on
another env.

I suggest to keep it simple for now as we have the issue mentioned by
Evgeny: fuel client is supposed to work from other nodes, and we will need
additional verification code in there. Also, to make it smooth, we will have
to end up with a few more checks - like what if tarball is broken, what if
we can't find install script in it, etc.
I'd suggest to run it simple for 6.0, and then we will see how it's being
used and what other limitations / issues we have around plugin installation
and usage. We can consider to make this functionality as part of fuel client
a bit later.

Thanks,

On Tue, Oct 21, 2014 at 6:57 PM, Vitaly Kramskikh
wrote:

Hi,

As for a separate section for plugins, I think we should not force it and
leave this decision to a plugin developer, so he can create just a single
checkbox or a section of the settings tab or a separate tab depending on
plugin functionality. Plugins should be able to modify arbitrary release
fields. For example, if Ceph was a plugin, it should be able to extend
wizard config to add new options to Storage pane. If vCenter was a plugin,
it should be able to set maximum amount of Compute nodes to 0.

2014-10-20 21:21 GMT+07:00 Evgeniy L :

Hi guys,

Romans' questions:

I feel like we should not require user to unpack the plugin before
installing it.
Moreover, we may chose to distribute plugins in our own format, which
we
may potentially change later. E.g. "lbaas-v2.0.fp".

I like the idea of putting plugin installation functionality in fuel
client, which is installed
on master node.
But in the current version plugin installation requires files operations
on the master,
as result we can have problems if user's fuel-client is installed on
another env.
What we can do is to try to determine where fuel-client is installed, if
it's master
node, we can perform installation, if it isn't master node, we can show
user the
message, that in the current version remote plugin installation is not
supported.
In the next versions if we implement plugin manager (which is separate
service
for plugins management) we will be able to do it remotely.

How are we planning to distribute fuel plugin builder and its
updates?

Yes, as Mike mentioned our plan is to release it on PyPi which is python
packages
repository, so any developer will be able to run pip install fpb and
get the tool.

What happens if an error occurs during plugin installation?

Plugins installation process is very simple, our plan is to have some
kind of transaction,
to make it atomic.

  1. register plugin via API
  2. copy the files

In case of error on the 1st step, we can do nothing, in case of error on
the 2nd step,
remove files if there are any, and delete a plugin via rest api. And
show user a message.

What happens if an error occurs during plugin execution?

In the first iteration we are going to interrupt deployment if there are
any errors for plugin's
tasks, also we are thinking how to improve it, for example we wanted to
provide a special
flag for each task, like faildeploumenton_error, and only if it's
true, we fail deployment in
case of failed task. But it can be tricky to implement, it requires to
change the current
orchestrator/nailgun error handling logic. So, I'm not sure if we can
implement this logic in
the first release.

Regarding to meaningful error messages, yes, we want to show the user,
which plugin
causes the error.

Shall we consider a separate place in UI (tab) for plugins?

+1 to Mike's answer

When are we planning to focus on the 2 plugins which were identified
as must-haves
for 6.0? Cinder & LBaaS

For Cinder we are going to implement plugin which configures GlusterFS
as cinder backend,
so, if user has installed GlusterFS cluster, we can configure our cinder
to work with it,
I want to mention that we don't install GlusterFS nodes, we just
configure cinder to work
with user's GlusterFS cluster.
Stanislaw B. already did some scripts which configures cinder to work
with GlusterFS, so
we are on testing stage.

Regarding to LBaaS, Stanislaw B. did multinode implementation, ha
implementation is tricky
and requires some additional work, we are working on it.

Nathan's questions:

Looks like Mike answered UI related questions.

Do we offer any kind of validation for settings on plug-ins? Or
some way to for the developer
to ensure that setting that cannot be default or computed get
requested for the plug-in?

Yes, each field can have regexp which is used during the validation.

Mike's questions:

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I
understand it might sound like a bikeshedding now, but I believe if
we develop it right from
the very beginning, then we can save some time later. So I would
suggest normal, short INFO
output, and verbose one with --debug.

Agree.

Thanks for your feedback,

On Sun, Oct 19, 2014 at 1:11 PM, Mike Scherbakov
wrote:

Hi all,
I moved this conversation to openstack-dev to get a broader audience,
since we started to discuss technical details.

Raw notes from demo session:
https://etherpad.openstack.org/p/cinder-neutron-plugins-second-demo.

Let me start answering on a few questions below from Roman & Nathan.

How are we planning to distribute fuel plugin builder and its updates?
Ideally, it should be available externally (outside of master node). I don't
want us to repeat the same mistake as we did with Fuel client, which doesn't
seem to be usable as an external dependency.

The plan was to have Fuel Plugin Builder (fpb) on PyPI. Ideally it
should be backward compatible with older Fuel release, i.e. when there is
Fuel 7.0 out, you should be still able to create plugin for Fuel 6.0. If
that it is going to be overcomplicated - I suggested to produce fpb for
every Fuel release, and name it like fpb60, fpb61, fpb70, etc. Then it
becomes easier to support and maintain plugin builders for certain versions
of Fuel.
Speaking about Fuel Client - there is no mistake. It's been discussed
dozens of times, it's just lack of resources to make it on PyPI as well as
to fix a few other things. I hope it could be done as part of efforts from
[2].

  • Perhaps we have a separate settings tab just for Plug-Ins? For
    some complex plug-ins, they might require a dedicated tab. If we have too
    many tabs it could get messy.

Shall we consider a separate place in UI (tab) for plugins? Settings
tab seems to be overloaded.

This is certainly under planning and discussion for future releases.
See [1], for example. For 6.0, we agreed that we can just extend existing
Settings tab with plugins-related fields.

One minor thing from me, which I forgot to mention during the demo:
verbosity of fpb run. I understand it might sound like a bikeshedding now,
but I believe if we develop it right from the very beginning, then we can
save some time later. So I would suggest normal, short INFO output, and
verbose one with --debug.

Thanks for feedback folks!!!

[1]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37196.html
[2]
https://www.mail-archive.com/openstack-dev at lists.openstack.org/msg37001.html

---------- Forwarded message ----------
From: Nathan Trueblood
Date: Sat, Oct 18, 2014 at 3:24 AM
Subject: Re: plugins

Agreed - I thought this initial PoC was great.

A few initial thoughts about settings in the UI and plug-in in general:

  • Perhaps we have a separate settings tab just for Plug-Ins? For
    some complex plug-ins, they might require a dedicated tab. If we have too
    many tabs it could get messy.
  • It seems like we should consider how we handle the VMWare settings in
    light of plug-ins as well. Since with VMWare we have a lot of setting to
    configure and settings validation.
  • Do we offer any kind of validation for settings on plug-ins? Or
    some way to for the developer to ensure that setting that cannot be default
    or computed get requested for the plug-in?

  • We need to think carefully about both the plug-in developer
    experience (how hard to test, get error messages, etc) and the experience
    for the user who deploys the plug-in into an environment.

-Nathan

On Fri, Oct 17, 2014 at 4:13 PM, Roman Alekseenkov
wrote:

I watched both videos (creating a file with the text from UI &&
installing and starting a service).

It looks pretty good!! Some initial feedback/questions:

I like the fact that fuel plugin builder appends version to the name
and makes it "fuel-awesome-plugin-1.2.3.tar". The approach is similar to
Java/Maven and is a good one.
I feel like we should not require user to unpack the plugin before
installing it. Moreover, we may chose to distribute plugins in our own
format, which we may potentially change later. E.g. "lbaas-v2.0.fp". I'd
rather stick with two actions:

Assembly (externally): fpb --build
Installation (on master node): fuel --install-plugin

How are we planning to distribute fuel plugin builder and its updates?
Ideally, it should be available externally (outside of master node). I don't
want us to repeat the same mistake as we did with Fuel client, which doesn't
seem to be usable as an external dependency.
How do we handle errors?

What happens if an error occurs during plugin installation?
What happens if an error occurs during plugin execution? Does it
(should it?) fail the deployment? Will we show user an error message with
the name of plugin that failed?

Shall we consider a separate place in UI (tab) for plugins? Settings
tab seems to be overloaded.
When are we planning to focus on the 2 plugins which were identified
as must-haves for 6.0? Cinder & LBaaS

Once again, great job guys!

Thanks,
Roman

On Fri, Oct 17, 2014 at 9:32 AM, Mike Scherbakov
wrote:

Thanks, Evgeny, excellent work!
Roman, I believe we are "green" with the feature. Watch yourself.

Mike Scherbakov

mihgen

On Oct 17, 2014 8:25 PM, "Evgeniy L" wrote:

--
Mike Scherbakov

mihgen


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Vitaly Kramskikh,
Software Engineer,
Mirantis, Inc.


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Mike Scherbakov

mihgen


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Dmitry Borodaenko

responded Oct 23, 2014 by Dmitry_Borodaenko (7,240 points)   1 3 5
...