settingsLogin | Registersettings

[openstack-dev] [horizon] packaging problem production build question

0 votes

I've been going over the packaging problem in an effort to see how we can
move to something better. Given the current proposal around bower I'm still
left with a production deployment question.

For a build environment sitting in isolation, unable to download from the
Internet including Github, how would they be able to get all the bower
controlled packages to create a system horizon package (e.g., rpm or deb)?

These build environments currently use mirrors and controlled packages. For
example, someone might have a pypi mirror with copies of the xstatic
packages. This is tightly controlled. If bower is managing packages where,
in theory, would it get them from for an environment like this?

I may have missed something. If this has already been answered please
excuse me and point me in the right direction.

Thanks,
Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openstack.org/pipermail/openstack-dev/attachments/20150108/8a1e2d58/attachment.html

asked Jan 8, 2015 in openstack-dev by Matthew_Farina (2,160 points)   1 4
retagged Jan 28, 2015 by admin

5 Responses

0 votes

Bower is not for use in production environments. There will continue to be
two environment setup procedures, as there are today. For production,
deploy Horizon and its dependencies via system packages. For development
and testing leverage bower to pull the javascript resources, much as pip is
used today and continue to use pip for python dependencies.

For those running CI environments, remote access will likely be required
for bower to work. Although, it seems something like private-bower [1]
could be utilized to leverage a local mirror where access or network
performance are issues.

David

[1] https://www.npmjs.com/package/private-bower

On Thu, Jan 8, 2015 at 2:28 PM, Matthew Farina wrote:

I've been going over the packaging problem in an effort to see how we can
move to something better. Given the current proposal around bower I'm still
left with a production deployment question.

For a build environment sitting in isolation, unable to download from the
Internet including Github, how would they be able to get all the bower
controlled packages to create a system horizon package (e.g., rpm or deb)?

These build environments currently use mirrors and controlled packages.
For example, someone might have a pypi mirror with copies of the xstatic
packages. This is tightly controlled. If bower is managing packages where,
in theory, would it get them from for an environment like this?

I may have missed something. If this has already been answered please
excuse me and point me in the right direction.

Thanks,
Matt


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Jan 8, 2015 by David_Lyle (2,780 points)   2 3
0 votes

On 2015-01-08 15:11:24 -0700 (-0700), David Lyle wrote:
[...]
For those running CI environments, remote access will likely be
required for bower to work. Although, it seems something like
private-bower [1] could be utilized to leverage a local mirror
where access or network performance are issues.
[...]

There's a very good chance we'll want to do something similar for
the official OpenStack CI jobs as well. We already go to extreme
lengths to pre-cache and locally mirror things which software would
otherwise try to retrieve from random parts of the Internet during
setup for tests. If your software retrieves files from 10 random
places over the network, the chances of your job failing because of
one of them being offline is multiplied by 10. As that number grows,
so grows your lack of testability.
--
Jeremy Stanley

responded Jan 8, 2015 by Jeremy_Stanley (56,700 points)   3 5 7
0 votes

Thanks for humoring me as I ask these questions. I'm just trying to connect
the dots.

How would system packages work in practice? For example, when it comes to
ubuntu lucid (10.04 LTS) there is no system package meeting the jQuery
requirement and for precise (12.04 LTS) you need precise-backports. This is
for the most popular JavaScript library. There is only an angular package
for trusty (14.04 LTS) and the version is older than the horizon minimum.

private-bower would be a nice way to have a private registry. But, bower
packages aren't packages in the same sense as system or pypi packages. If I
understand it correctly, when bower downloads something it doesn't get it
from the registry (bower.io or private-bower). Instead it goes to the
source (e.g., Github) to download the code. private-bower isn't a package
mirror but instead a private registry (of location). How could
private-bower be used to negate network effects if you still need to go out
to the Internet to get the packages?

On Thu, Jan 8, 2015 at 5:11 PM, David Lyle wrote:

Bower is not for use in production environments. There will continue to be
two environment setup procedures, as there are today. For production,
deploy Horizon and its dependencies via system packages. For development
and testing leverage bower to pull the javascript resources, much as pip is
used today and continue to use pip for python dependencies.

For those running CI environments, remote access will likely be required
for bower to work. Although, it seems something like private-bower [1]
could be utilized to leverage a local mirror where access or network
performance are issues.

David

[1] https://www.npmjs.com/package/private-bower

On Thu, Jan 8, 2015 at 2:28 PM, Matthew Farina
wrote:

I've been going over the packaging problem in an effort to see how we can
move to something better. Given the current proposal around bower I'm still
left with a production deployment question.

For a build environment sitting in isolation, unable to download from the
Internet including Github, how would they be able to get all the bower
controlled packages to create a system horizon package (e.g., rpm or deb)?

These build environments currently use mirrors and controlled packages.
For example, someone might have a pypi mirror with copies of the xstatic
packages. This is tightly controlled. If bower is managing packages where,
in theory, would it get them from for an environment like this?

I may have missed something. If this has already been answered please
excuse me and point me in the right direction.

Thanks,
Matt


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Jan 8, 2015 by Matthew_Farina (2,160 points)   1 4
0 votes

On 08/01/15 23:46, Matthew Farina wrote:
Thanks for humoring me as I ask these questions. I'm just trying to
connect the dots.

How would system packages work in practice? For example, when it comes
to ubuntu lucid (10.04 LTS) there is no system package meeting the
jQuery requirement and for precise (12.04 LTS) you need
precise-backports. This is for the most popular JavaScript library.
There is only an angular package for trusty (14.04 LTS) and the version
is older than the horizon minimum.

private-bower would be a nice way to have a private registry. But, bower
packages aren't packages in the same sense as system or pypi packages.
If I understand it correctly, when bower downloads something it doesn't
get it from the registry (bower.io or private-bower).
Instead it goes to the source (e.g., Github) to download the code.
private-bower isn't a package mirror but instead a private registry (of
location). How could private-bower be used to negate network effects if
you still need to go out to the Internet to get the packages?

For a deployment, you want updates, often installed automatically.

Your repository providing your horizon package needs to provide required
dependencies as well.

I wouldn't recommend to use bower. In some environments, it's not
allowed to use third party repositories at all. A test environment
should match a possible production environment, where it can. This one
is quite easy.

Matthias

responded Jan 9, 2015 by Matthias_Runge (6,700 points)   1 2 3
0 votes

-----Original Message-----
From: Jeremy Stanley [mailto:fungi at yuggoth.org]
Sent: 08 January 2015 22:26
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [horizon] packaging problem production
build question

On 2015-01-08 15:11:24 -0700 (-0700), David Lyle wrote:
[...]

For those running CI environments, remote access will likely be
required for bower to work. Although, it seems something like
private-bower [1] could be utilized to leverage a local mirror where
access or network performance are issues.
[...]

There's a very good chance we'll want to do something similar for the
official OpenStack CI jobs as well. We already go to extreme lengths to
pre-cache and locally mirror things which software would otherwise try
to retrieve from random parts of the Internet during setup for tests. If
your software retrieves files from 10 random places over the network,
the chances of your job failing because of one of them being offline is
multiplied by 10. As that number grows, so grows your lack of
testability.

Local mirrors are also used to control the version of software included in a build, so that builds can be repeatable independently of changes to external sources. Is this supported by bower?

--
Jeremy Stanley


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Thanks,
Jon-Paul Sullivan ? Cloud Services - @hpcloud

Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park, Galway.
Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John Rogerson's Quay, Dublin 2.
Registered Number: 361933

The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender.

To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as "HP CONFIDENTIAL".

responded Jan 9, 2015 by Sullivan,_Jon_Paul (1,140 points)   1 2
...