settingsLogin | Registersettings

Re: [openstack-dev] [nova][neutron] How do you use the instance IP filter?

0 votes

On Thu, Oct 26, 2017 at 09:23:50PM -0500, Matt Riedemann wrote:

Nova has had this long-standing known performance issue if you're filtering
a large number of instances by IP. The instance IPs are stored in a JSON
blob in the database so we don't do filtering in SQL. We pull the instances
out of the database, deserialize the JSON and then apply a regex filter
match in the nova-api python code.

At the Queens PTG we talked about possible ways to fix this and came up with
this nova spec:

https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/improve-filter-instances-by-ip-performance.html

The idea is to have nova get ports from neutron and apply the IP filter in
neutron to whittle down the ports, then from that list of ports get the
instances to pull out of the nova database.

One issue that has come up with this is neutron does not currently support
regex filters when listing ports. There is an RFE for adding that:

https://bugs.launchpad.net/neutron/+bug/1718605

The proposed neutron implementation is to just do SQL LIKE substring
matching in the database.

Can you use RLIKE/REGEX? or is that too MySQL specific ?

Yours Tony.


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

asked Nov 2, 2017 in openstack-dev by Tony_Breeds (19,660 points)   3 6 10

3 Responses

0 votes

On 10/26/2017 9:54 PM, Tony Breeds wrote:
Can you use RLIKE/REGEX? or is that too MySQL specific ?

I thought about that, and my gut response is 'no' because even if it
does work for mysql, I'm assuming regex pattern matching for postgresql
is different. And then you have different API behavior between clouds
based on the backend database they are using, and now we've opened that
whole can of worms again.

--

Thanks,

Matt


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 27, 2017 by mriedemos_at_gmail.c (15,720 points)   2 4 7
0 votes

On Thu, Oct 26, 2017 at 10:35:47PM -0500, Matt Riedemann wrote:
On 10/26/2017 9:54 PM, Tony Breeds wrote:

Can you use RLIKE/REGEX? or is that too MySQL specific ?

I thought about that, and my gut response is 'no' because even if it does
work for mysql, I'm assuming regex pattern matching for postgresql is
different. And then you have different API behavior between clouds based on
the backend database they are using, and now we've opened that whole can of
worms again.

Yeah:
column.op('rlike')() # Mysql
column.op('~')() # Pgsql

I have no idea if the regexs themselves would be compatible and of
course there are other RDBMSs

Yours Tony.


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

responded Oct 27, 2017 by Tony_Breeds (19,660 points)   3 6 10
0 votes

FYI, Nova did use regex
https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L2408

2017-10-27 11:35 GMT+08:00 Matt Riedemann mriedemos@gmail.com:

On 10/26/2017 9:54 PM, Tony Breeds wrote:

Can you use RLIKE/REGEX? or is that too MySQL specific ?

I thought about that, and my gut response is 'no' because even if it does
work for mysql, I'm assuming regex pattern matching for postgresql is
different. And then you have different API behavior between clouds based on
the backend database they are using, and now we've opened that whole can of
worms again.

--

Thanks,

Matt


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Nov 2, 2017 by Alex_Xu (8,960 points)   1 3 3
...