settingsLogin | Registersettings

[openstack-dev] [requirements] pycrypto is dead, long live pycryptodome... or cryptography...

0 votes

So, pycrypto upstream is dead and has been for a while, we should look
at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth, keystonemiddleware,
kolla, openstack-ansible, and a couple of other smaller places.

Development of it was forked into pycryptodome, which is supposed to be
a drop in replacement. The problem is that due to co-installability
requirements we can't have half of packages out there using pycrypto and
the other half using pycryptodome. We'd need to hard switch everyone as
both packages install into the same namespace.

Another alternative would be to use something like cryptography instead,
though it is not a drop in replacement, the migration would be able to
be done piecemeal.

I'd be interested in hearing about migration plans, especially from the
affected projects.

--
Matthew Thode (prometheanfire)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

asked Mar 29, 2017 in openstack-dev by prometheanfire_at_ge (6,880 points)   1 4 5

12 Responses

0 votes

Matthew,

Please see the last time i took inventory:
https://review.openstack.org/#/q/pycryptodome+owner:dims-v

Thanks,
Dims

On Wed, Mar 8, 2017 at 2:03 PM, Matthew Thode prometheanfire@gentoo.org wrote:
So, pycrypto upstream is dead and has been for a while, we should look
at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth, keystonemiddleware,
kolla, openstack-ansible, and a couple of other smaller places.

Development of it was forked into pycryptodome, which is supposed to be
a drop in replacement. The problem is that due to co-installability
requirements we can't have half of packages out there using pycrypto and
the other half using pycryptodome. We'd need to hard switch everyone as
both packages install into the same namespace.

Another alternative would be to use something like cryptography instead,
though it is not a drop in replacement, the migration would be able to
be done piecemeal.

I'd be interested in hearing about migration plans, especially from the
affected projects.

--
Matthew Thode (prometheanfire)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Davanum Srinivas :: https://twitter.com/dims


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 8, 2017 by Davanum_Srinivas (35,920 points)   2 4 9
0 votes

I'm aware, iirc it was brought up when pysaml2 had to be fixed due to a
CVE. This thread is more looking for a long term fix.

On 03/08/2017 01:11 PM, Davanum Srinivas wrote:
Matthew,

Please see the last time i took inventory:
https://review.openstack.org/#/q/pycryptodome+owner:dims-v

Thanks,
Dims

On Wed, Mar 8, 2017 at 2:03 PM, Matthew Thode prometheanfire@gentoo.org wrote:

So, pycrypto upstream is dead and has been for a while, we should look
at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth, keystonemiddleware,
kolla, openstack-ansible, and a couple of other smaller places.

Development of it was forked into pycryptodome, which is supposed to be
a drop in replacement. The problem is that due to co-installability
requirements we can't have half of packages out there using pycrypto and
the other half using pycryptodome. We'd need to hard switch everyone as
both packages install into the same namespace.

Another alternative would be to use something like cryptography instead,
though it is not a drop in replacement, the migration would be able to
be done piecemeal.

I'd be interested in hearing about migration plans, especially from the
affected projects.

--
Matthew Thode (prometheanfire)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Matthew Thode (prometheanfire)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

responded Mar 8, 2017 by prometheanfire_at_ge (6,880 points)   1 4 5
0 votes

Ack thanks Matthew!

On Wed, Mar 8, 2017 at 2:24 PM, Matthew Thode prometheanfire@gentoo.org wrote:
I'm aware, iirc it was brought up when pysaml2 had to be fixed due to a
CVE. This thread is more looking for a long term fix.

On 03/08/2017 01:11 PM, Davanum Srinivas wrote:

Matthew,

Please see the last time i took inventory:
https://review.openstack.org/#/q/pycryptodome+owner:dims-v

Thanks,
Dims

On Wed, Mar 8, 2017 at 2:03 PM, Matthew Thode prometheanfire@gentoo.org wrote:

So, pycrypto upstream is dead and has been for a while, we should look
at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth, keystonemiddleware,
kolla, openstack-ansible, and a couple of other smaller places.

Development of it was forked into pycryptodome, which is supposed to be
a drop in replacement. The problem is that due to co-installability
requirements we can't have half of packages out there using pycrypto and
the other half using pycryptodome. We'd need to hard switch everyone as
both packages install into the same namespace.

Another alternative would be to use something like cryptography instead,
though it is not a drop in replacement, the migration would be able to
be done piecemeal.

I'd be interested in hearing about migration plans, especially from the
affected projects.

--
Matthew Thode (prometheanfire)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Matthew Thode (prometheanfire)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Davanum Srinivas :: https://twitter.com/dims


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 8, 2017 by Davanum_Srinivas (35,920 points)   2 4 9
0 votes

One of my goals for Barbican for this cycle is to migrate our code to use pyca/cryptography exclusively. We currently depend on both because at one point we needed things that were not available in early releases of cryptography.

  • Douglas Mendizábal (redrobot)

On Mar 8, 2017, at 1:11 PM, Davanum Srinivas davanum@gmail.com wrote:

Matthew,

Please see the last time i took inventory:
https://review.openstack.org/#/q/pycryptodome+owner:dims-v

Thanks,
Dims

On Wed, Mar 8, 2017 at 2:03 PM, Matthew Thode prometheanfire@gentoo.org wrote:

So, pycrypto upstream is dead and has been for a while, we should look
at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth, keystonemiddleware,
kolla, openstack-ansible, and a couple of other smaller places.

Development of it was forked into pycryptodome, which is supposed to be
a drop in replacement. The problem is that due to co-installability
requirements we can't have half of packages out there using pycrypto and
the other half using pycryptodome. We'd need to hard switch everyone as
both packages install into the same namespace.

Another alternative would be to use something like cryptography instead,
though it is not a drop in replacement, the migration would be able to
be done piecemeal.

I'd be interested in hearing about migration plans, especially from the
affected projects.

--
Matthew Thode (prometheanfire)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Davanum Srinivas :: https://twitter.com/dims


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 8, 2017 by Douglas_Mendizabal (3,400 points)   5 5
0 votes

On Wed, Mar 8, 2017 at 1:03 PM, Matthew Thode prometheanfire@gentoo.org
wrote:

So, pycrypto upstream is dead and has been for a while, we should look
at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth, keystonemiddleware,
kolla, openstack-ansible, and a couple of other smaller places.

keystoneauth didn't actually use pycrypto even though it was in
test-requirements.txt, so I posted a change to remove it:
https://review.openstack.org/#/c/443318/

  • Brant

Development of it was forked into pycryptodome, which is supposed to be
a drop in replacement. The problem is that due to co-installability
requirements we can't have half of packages out there using pycrypto and
the other half using pycryptodome. We'd need to hard switch everyone as
both packages install into the same namespace.

Another alternative would be to use something like cryptography instead,
though it is not a drop in replacement, the migration would be able to
be done piecemeal.

I'd be interested in hearing about migration plans, especially from the
affected projects.

--
Matthew Thode (prometheanfire)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
- Brant


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 8, 2017 by Brant_Knudson (5,640 points)   1 2 2
0 votes

On Wed, Mar 8, 2017 at 8:03 PM, Matthew Thode prometheanfire@gentoo.org wrote:
So, pycrypto upstream is dead and has been for a while, we should look
at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth, keystonemiddleware,
kolla, openstack-ansible, and a couple of other smaller places.

Heat keeps it mostly for (old) backward compatibility. We can possibly
remove it now, especially if it helps global requirements.

--
Thomas


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 8, 2017 by therve_at_redhat.com (2,620 points)   1 2
0 votes

Sounds like a good candidate for a cross-project release goal.

A non-controversial situation, the work is a no-op for most, a specific
deliverable for a few, and a mechanism to close the loop and make sure it
gets done in a specific timeframe?

Thanks for surfacing it Matthew.

-amrith

-----Original Message-----
From: Davanum Srinivas [mailto:davanum@gmail.com]
Sent: Wednesday, March 8, 2017 2:30 PM
To: OpenStack Development Mailing List (not for usage questions)

Subject: Re: [openstack-dev] [requirements] pycrypto is dead, long live
pycryptodome... or cryptography...

Ack thanks Matthew!

On Wed, Mar 8, 2017 at 2:24 PM, Matthew Thode prometheanfire@gentoo.org
wrote:
I'm aware, iirc it was brought up when pysaml2 had to be fixed due to
a CVE. This thread is more looking for a long term fix.

On 03/08/2017 01:11 PM, Davanum Srinivas wrote:

Matthew,

Please see the last time i took inventory:
https://review.openstack.org/#/q/pycryptodome+owner:dims-v

Thanks,
Dims

On Wed, Mar 8, 2017 at 2:03 PM, Matthew Thode prometheanfire@gentoo.org
wrote:

So, pycrypto upstream is dead and has been for a while, we should
look at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth,
keystonemiddleware, kolla, openstack-ansible, and a couple of other
smaller places.

Development of it was forked into pycryptodome, which is supposed to
be a drop in replacement. The problem is that due to
co-installability requirements we can't have half of packages out
there using pycrypto and the other half using pycryptodome. We'd
need to hard switch everyone as both packages install into the same
namespace.

Another alternative would be to use something like cryptography
instead, though it is not a drop in replacement, the migration would
be able to be done piecemeal.

I'd be interested in hearing about migration plans, especially from
the affected projects.

--
Matthew Thode (prometheanfire)


______ OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Matthew Thode (prometheanfire)


____ OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Davanum Srinivas :: https://twitter.com/dims


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 8, 2017 by amrith.kumar_at_gmai (3,580 points)   2 3
0 votes

On 03/08/2017 05:38 PM, Amrith Kumar wrote:
Sounds like a good candidate for a cross-project release goal.

A non-controversial situation, the work is a no-op for most, a specific
deliverable for a few, and a mechanism to close the loop and make sure it
gets done in a specific timeframe?

Thanks for surfacing it Matthew.

-amrith

Heh, thanks, I suspect a few things are fairly cross project for
requirements. Moving to new webob / eventlet / sqlalchemy for instance.

Is there a specific process we go through to do these?

--
Matthew Thode (prometheanfire)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

responded Mar 9, 2017 by prometheanfire_at_ge (6,880 points)   1 4 5
0 votes

On 3/8/17 2:03 PM, Matthew Thode wrote:
So, pycrypto upstream is dead and has been for a while, we should look
at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth, keystonemiddleware,
kolla, openstack-ansible, and a couple of other smaller places.

[snip]

I'd be interested in hearing about migration plans, especially from the
affected projects.

Glance report:
- pycrypto isn't used in glance_store or python-glanceclient
- Glance already uses cryptography for image-signature verification, so
our path will be to migrate from pycrypto -> cryptography
- I've got a patch up for this: https://review.openstack.org/#/c/449401/

cheers,
brian


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 29, 2017 by rosmaita.fossdev_at_ (4,180 points)   1 2 2
0 votes

With pycrypto removed from keystoneauth [0] (thanks Brant, Monty, and
Morgan!), I did some poking at the usage in keystonemiddleware [1].

The usage is built into auth_token middleware for encrypting and decrypting
things stored in cache [2], but it is conditional based on configuration
[3] and whether or not pycrypto is installed [4]. The encryption of things
before caching them is disabled by default.

We've also had several discussions about moving keystonemiddleware to using
oslo.cache instead of it's own caching implementation [5] for py3 reasons.
If we're going to invest time into making that switch, grouping the switch
from pycrypto to pyca/cryptography doesn't sound unreasonable.

Any thoughts on this from a keystone perspective? I can try and work them
into a spec proposal for keystonemiddleware since I'll be proposing one for
the oslo.cache switch [6].

[0] https://review.openstack.org/#/c/443318/
[1]
https://github.com/openstack/keystonemiddleware/blob/a2e3d60644aadb4ecb3d49dadbcd5d4c1dec2176/test-requirements.txt#L12
[2]
https://github.com/openstack/keystonemiddleware/blob/a2e3d60644aadb4ecb3d49dadbcd5d4c1dec2176/keystonemiddleware/auth_token/_memcache_crypt.py#L19-L21
[3]
https://github.com/openstack/keystonemiddleware/blob/a2e3d60644aadb4ecb3d49dadbcd5d4c1dec2176/keystonemiddleware/auth_token/_opts.py#L109-L122
[4]
https://github.com/openstack/keystonemiddleware/blob/a2e3d60644aadb4ecb3d49dadbcd5d4c1dec2176/keystonemiddleware/auth_token/_memcache_crypt.py#L42-L46
[5]
http://eavesdrop.openstack.org/meetings/keystone/2017/keystone.2017-03-21-18.00.log.html#l-136
[6]
http://eavesdrop.openstack.org/meetings/keystone/2017/keystone.2017-03-21-18.00.log.html#l-149

On Wed, Mar 29, 2017 at 9:56 AM, Brian Rosmaita rosmaita.fossdev@gmail.com
wrote:

On 3/8/17 2:03 PM, Matthew Thode wrote:

So, pycrypto upstream is dead and has been for a while, we should look
at moving off of it for both bugfix and security reasons.

Currently it's used by the following.

barbican, cinder, trove, glance, heat, keystoneauth, keystonemiddleware,
kolla, openstack-ansible, and a couple of other smaller places.

[snip]

I'd be interested in hearing about migration plans, especially from the
affected projects.

Glance report:
- pycrypto isn't used in glance_store or python-glanceclient
- Glance already uses cryptography for image-signature verification, so
our path will be to migrate from pycrypto -> cryptography
- I've got a patch up for this: https://review.openstack.org/#/c/449401/

cheers,
brian


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 29, 2017 by Lance_Bragstad (11,080 points)   2 3 6
...