On Thu, Aug 17, 2017 at 4:24 PM, Luke Browning LukeBrowning@us.ibm.com
I understand that the Trove project will be going into maintenance mode
soon. I have a number of Trove related enhancements and bug fixes that I
would like to submit to the community, but I don't know if they would be
considered given that the project is going into maintenance mode. Please
elaborate on what you mean by maintenance mode.
It is not a foregone conclusion that the project will go into maintenance
mode. Thierry and I (for example) are not convinced that this is the right
course of action, but if there are no offers to contribute to the project,
it may be a decision by default.
Do you subscribe to the OpenStack-dev mailing list. Please also see
Finally, for a description of what maintenance mode means, please refer:
Would Trove be updated to support new OpenStack versions?
Would support be provided for Trove gate testing?
Would elements be enhanced to support Xenial and newer versions of
Is there a time limit associated with maintenance mode? For example, would
it remain in effect for a year or two after the new stuff is released?
For the four questions above, I direct your attention to the definition
of the maintenance-mode tag (https://governance.openstack.
org/tc/reference/tags/status_maintenance-mode.html) and remind you that
support for different versions from the community, for gate testing and for
elements is dependent on people participating (and their employers allowing
them to do this). At this point, I have no one who has stepped up to do
this in any significant way. There are a couple of people from China Mobile
who want to help but who are largely in the weeds, trying to fix this and
that bug but have no idea what Trove is all about. IBM has a core reviewer
on the project (Mariam is copied on this email). I am happy that she's
able to devote what time she can to the project but absent competent
reviewers, whether your changes ever get merged or not remains an
interesting question. Since you will be contributing elements and propose
to contribute a tool, will you be (or will IBM be) offering to support it?
Let me provide some back ground on my code changes, so you will have a
better idea of what might be submitted in a patch.
I have written a fully automated command that creates a virtual guest
image based on Ubuntu 16.04 containing a user specified version of a
database. The user's selection is validated against an internal list of
databases built into the tool. Once validated, the tool creates the image,
registers it with Glance, and then creates a Trove Datastore definition for
it. This is done in a single command invocation that typically takes about
25 minutes to complete. For some of the databases, it takes considerably
longer (2 hours for mongodb) as I have to compile source code. The guest
image that is produced is complete from a Trove perspective. It includes
the Cloud specific trove-guestagent.conf file and SSH public keys for the
DBA and OpenStack controller nodes.
What mechanism does it use to develop the image? is it diskimage-builder
or some other?
Is there something the matter with the existing tool that exists, it
already works, it does more than just build an image, and it is already
integrated in the gate. Why not use that?
This approach is nice (having a full guest image with the guest agent and
all that but as you observe, it takes about 2 hours per build and from a
developers perspective, this is a pain.
I assume that the tool is therefore for production use cases.
Could the elements that you have developed be used with the existing tool
or is your tool a complete replacement for the one that already exists?
Is there some reason that this tool was not developed in the open, using
the normal open development process?
There are ways to accelerate the compiling of source code and such through
the use of image layers (if you are using DIB). I have elements and code
that will build most of the trove elements in minutes and I'm waiting to
see whether there is any point in contributing them to the upstream or not,
at this point. I could show you how to do that, and it would be awesome to
have you build your tool into the build process somehow. That would be a
win-win for all ...
I also have commands that setup Nova flavors on a per database basis.
These flavors can be customized by the user before they are instantiated in
The existing command (trovestack) already does this.
I wouldn't expect to drop these commands to the Trove community. Only the
changes to the Trove-guestagent and elements enabling databases.
Here's the list of databases that are currently supported for ppc64le. In
some cases, I have had to add Xenial support.
mariadb 10.1 --- package comes from the community
mongodb 3.4 community edition --- source code from the community is
compiled as there is no ppc64le package
mongodb 3.4 enterprise edition --- package comes from community
mysql 5.7 --- package comes from Ubuntu 16.04. percona-xtrabackup-24 is
compiled as there is no power support in community
postgresql 9.5 --- package comes from Ubuntu 16.04
postgresql 9.6 --- package comes from community
redis 3.0 --- package comes from Ubuntu 16.04
redis 3.2 --- source code from the community is compiled
Why not just contribute the elements to the existing project and have the
current tool use them?
What of other databases currently supported? galera cluster (pxc),
vertica, db2, percona, couchdb, and couchbase?
My element related code changes add environment variables that control
the database version, the source of the database software (distro,
community, enterprise), and logging options. In this way, the logic in the
element is more generalized and can be made to support multiple versions. I
also added an environment variable that specifies the name of the source
package to be downloaded and compiled when that is applicable. I also have
environment variables to enable database and trove guest agent logging.
Which is nice except that your guest agent code may or may not know how
to deal with that.
My trove-guestagent code changes are mostly about not reconfiguring the
database with configuration data remotely passed to it from the task
manager. My code changes are isolated to the prepare step. This logic looks
something like: if database version x, add or remove configuration
parameter. Note there is relatively very little of this code, ~150 LOCs, as
database configuration parameters rarely change. Note there is already some
of this DB version specific logic in the trove guest agent, so I am just
adding more of it. But I believe a better way is not to alter the database
configuration data when the database is fully configured in a Trove
compliant manner via elements. This would be implemented through a new
command argument to the Trove Guest agent which says ignore configuration
changes. It would be set by the elements as well.
Hard to say whether this is a good approach or not with out more detail.
I haven't looked into the scale out configuration data yet, but this can
be handled differently as required.
Not sure what you mean.
So, are these the sorts of changes that I could add in a maintenance
Maintenance release of what, trove? Are you asking whether these changes
can be made in a stable branch? if yes, which branch? See
the section "Appropriate Fixes".
1) conditionally disable the dynamic installation and configuration of
database software in the trove-guest agent via a new command argument
2) add environment variables to trove elements enabling multiple versions
of databases to be installed and configured
3) add environment variables to trove elements to enable database and
trove-guest agent logging
4) add xenial support to elements
5) compiling source code in elements for missing stack elements for
These changes are implemented at: https://github.com/open-power-
OK, so this is a whole new tool, sure you can propose it but how do we
make sure that it gets integrated into the current testing/gate? If you
want to drop this then I'd also have to assume that there would be some
code that would make it part of the current gate, and ideally make it
generate the images for the current gate. But for that, the fact that you
install the guest agent onto the image is a serious drag.
Also, why this wasn't proposed as a blueprint or spec in the upstream so
it could be developed in the open, I don't quite understand. After all, you
are now showing up with a complete tool and asking that it be included
which largely runs counter to the whole notion of open development, at
least as I understand it.
Finally, once committed, do you (or IBM) commit to maintaining this code
and fixing bugs in it should they arise, and upgrade/update it from time to