settingsLogin | Registersettings

[openstack-dev] [glance] Does glance_store swift driver support range requests ?

0 votes

     

400 Bad Request


      Getting images randomly from this store is not supported. Offset:
0, length: 33


    
asked Nov 15, 2017 in openstack-dev by Matt_Keenan (260 points)   1

5 Responses

0 votes

I think it will a rather hard problem to solve. As swift store can be
configured to store objects in different configurations. I guess the next
question would be, what is your underlying problem -- multiple build
requests or is this for retry for a single download?

If the image is in image cache and you are hitting the glance node with
cached image (which is quite possible for tiny deployments), this feature
will be relatively easier.

On Mon, Nov 13, 2017 at 6:47 AM, Matt Keenan matt.keenan@oracle.com wrote:

Hi,

Just configured devstack on Fedora 26, and by default glance_store uses
swift for image storage. When attempting to get a specific range from a
glance stored image, it's reporting range requests are not supported e.g.:

$ curl -i -X GET -r 0-32 -H "X-Auth-Token: $auth_token"

http://10.169.104.255/image/v2/images/29b7aa
5e-3ec2-49b5-ab6b-d6cc5099f46c/file
HTTP/1.1 400 Bad Request
Date: Mon, 13 Nov 2017 10:43:23 GMT
Server: Apache/2.4.27 (Fedora) OpenSSL/1.1.0f-fips mod_wsgi/4.5.15
Python/2.7
Content-Length: 205
Content-Type: text/html; charset=UTF-8
x-openstack-request-id: req-5ed2239f-165b-406f-969b-5cc4ab8c632d
Connection: close

<html>
 <head>
  <title>400 Bad Request</title>
 </head>
 <body>
  <h1>400 Bad Request</h1>
  Getting images randomly from this store is not supported. Offset: 0,

length: 33


responded Nov 13, 2017 by nik.komawar_at_gmail (10,500 points)   2 2 3
0 votes

On 13/11/17 22:51, Nikhil Komawar wrote:
I think it will a rather hard problem to solve. As swift store can be
configured to store objects in different configurations. I guess the
next question would be, what is your underlying problem -- multiple
build requests or is this for retry for a single download?

If the image is in image cache and you are hitting the glance node
with cached image (which is quite possible for tiny deployments), this
feature will be relatively easier.

So the specific image stored in glance is a Unified Archive
(https://docs.oracle.com/cd/E36784_01/html/E38524/gmrlo.html).

During a UAR deployment the archive UUID is required and it is contained
in the first 33 characters of the UAR image, thus a range request for
this portion is required when initiating the deployment. Then the rest
of the archive is extracted and deployed.

I just want to know whether this capability is possible with swift ?

If I change the defaultstore in glanceapi.conf  to say "file" e.g.
default_store = file. restart devstack@g-api and then upload an archive.
The curl request succeeds, so file based range requests are working,
just the default swift setup is failing and I though it might be some
conf file setting was required to enable the capability. Pretty sure
cinder works (well it used to work last time I tried this which was with
way back with Mitaka :) ).

Anyhow if it's not supported then I can work around at least initially
using file store.

thanks

Matt

On Mon, Nov 13, 2017 at 6:47 AM, Matt Keenan <matt.keenan@oracle.com
matt.keenan@oracle.com> wrote:

Hi,

 Just configured devstack on Fedora 26, and by default
glance_store uses swift for image storage. When attempting to get
a specific range from a glance stored image, it's reporting range
requests are not supported e.g.:

    $ curl -i -X GET -r 0-32 -H "X-Auth-Token: $auth_token"
http://10.169.104.255/image/v2/images/29b7aa

    5e-3ec2-49b5-ab6b-d6cc5099f46c/file
    HTTP/1.1 400 Bad Request
    Date: Mon, 13 Nov 2017 10:43:23 GMT
    Server: Apache/2.4.27 (Fedora) OpenSSL/1.1.0f-fips
mod_wsgi/4.5.15 Python/2.7
    Content-Length: 205
    Content-Type: text/html; charset=UTF-8
    x-openstack-request-id: req-5ed2239f-165b-406f-969b-5cc4ab8c632d
    Connection: close

    <html>
     <head>
      <title>400 Bad Request</title>
     </head>
     <body>
      <h1>400 Bad Request</h1>
      Getting images randomly from this store is not supported.
Offset: 0, length: 33<br /><br />
     </body>

Upon investigation, glance-api log is emitting:

    Nov 13 10:45:31 devstack@g-api.service[22783]:
#033[01;31mERROR glance.location [#033[01;36mNone
req-ad6da3f0-ead1-486a-a873-d301f02b0888 #033[00;36mdemo
demo#033[01;31m]
#033[01;35m#033[0│·····················1;31mGlance tried all
active locations to get data for image
29b7aa5e-3ec2-49b5-ab6b-d6cc5099f46c but all have failed.#033[00m:
StoreRandomGetNotSupported: Getting images randomly from this
store is notMDg4OCAjMDMzWzAwOzM2bW supported. Offset: 0, length: 33

The exception StoreRandomGetNotSupported is emitted by
glance_store from glance_store/capabilities.py:

    op_exec_map = {
        'get': (exceptions.StoreRandomGetNotSupported
                if kwargs.get('offset') or
kwargs.get('chunk_size') else
                exceptions.StoreGetNotSupported),
        'add': exceptions.StoreAddDisabled,
        'delete': exceptions.StoreDeleteNotSupported}

Looking at _driver/swift/store.py I think range requests are
supported, it I've be unsuccessful in configuring it.

Does the glance_store swift driver support range requests ?

Can it be configured within a conf file, by somehow adding a
capability ?

thanks

Matt

-- 


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
--
Thanks,
Nikhil


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
HHGS : http://www.hh-gs.com


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Nov 15, 2017 by Matt_Keenan (260 points)   1
0 votes

On 15 November 2017 at 11:15, Matt Keenan matt.keenan@oracle.com wrote:
On 13/11/17 22:51, Nikhil Komawar wrote:

I think it will a rather hard problem to solve. As swift store can be
configured to store objects in different configurations. I guess the next
question would be, what is your underlying problem -- multiple build
requests or is this for retry for a single download?

If the image is in image cache and you are hitting the glance node with
cached image (which is quite possible for tiny deployments), this feature
will be relatively easier.

So the specific image stored in glance is a Unified Archive
(https://docs.oracle.com/cd/E36784_01/html/E38524/gmrlo.html).

During a UAR deployment the archive UUID is required and it is contained in
the first 33 characters of the UAR image, thus a range request for this
portion is required when initiating the deployment. Then the rest of the
archive is extracted and deployed.

Given the range you want is always at the beginning, is a range
request any different to doing a full get request and dripping the
connection when you've got the bytes you want?


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Nov 15, 2017 by Duncan_Thomas (16,160 points)   1 3 6
0 votes

On 11/15/2017 06:28 AM, Duncan Thomas wrote:
On 15 November 2017 at 11:15, Matt Keenan matt.keenan@oracle.com wrote:

On 13/11/17 22:51, Nikhil Komawar wrote:

I think it will a rather hard problem to solve. As swift store can be
configured to store objects in different configurations. I guess the next
question would be, what is your underlying problem -- multiple build
requests or is this for retry for a single download?

If the image is in image cache and you are hitting the glance node with
cached image (which is quite possible for tiny deployments), this feature
will be relatively easier.

So the specific image stored in glance is a Unified Archive
(https://docs.oracle.com/cd/E36784_01/html/E38524/gmrlo.html).

During a UAR deployment the archive UUID is required and it is contained in
the first 33 characters of the UAR image, thus a range request for this
portion is required when initiating the deployment. Then the rest of the
archive is extracted and deployed.

Given the range you want is always at the beginning, is a range
request any different to doing a full get request and dripping the
connection when you've got the bytes you want?

Or just store the UAR UUID in the image metadata...

-jay


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Nov 15, 2017 by Jay_Pipes (59,760 points)   3 6 13
0 votes

On 15 Nov 2017, at 7:40, Jay Pipes wrote:

On 11/15/2017 06:28 AM, Duncan Thomas wrote:

On 15 November 2017 at 11:15, Matt Keenan matt.keenan@oracle.com wrote:

On 13/11/17 22:51, Nikhil Komawar wrote:

I think it will a rather hard problem to solve. As swift store can be
configured to store objects in different configurations. I guess the next
question would be, what is your underlying problem -- multiple build
requests or is this for retry for a single download?

If the image is in image cache and you are hitting the glance node with
cached image (which is quite possible for tiny deployments), this feature
will be relatively easier.

So the specific image stored in glance is a Unified Archive
(https://docs.oracle.com/cd/E36784_01/html/E38524/gmrlo.html).

During a UAR deployment the archive UUID is required and it is contained in
the first 33 characters of the UAR image, thus a range request for this
portion is required when initiating the deployment. Then the rest of the
archive is extracted and deployed.

Given the range you want is always at the beginning, is a range
request any different to doing a full get request and dripping the
connection when you've got the bytes you want?

Or just store the UAR UUID in the image metadata...

-jay

Swift supports range requests (and multiple ranges at the same time)

eg: "Range: bytes=1-34, 100-1024"


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

responded Nov 15, 2017 by John_Dickinson (10,400 points)   2 3 5
...