settingsLogin | Registersettings

[Openstack-operators] Ocata provider network issue failing to raise interfaces

0 votes

Hi All,

I’m running into an issue with a new Ocata deployment. I’m unable to boot
an instance directly from the provider network. Instance logs show that it
keeps repeating “failed to raise interfaces” so it doesn’t get an IP
assigned, and then can’t reach the metatdata servers.

Booting an instance from a self-service network works fine, can ping other
internal IPs within the vlan, and can assign and use a floating IP.

Deployment is running linux bridge, with a flat network for provider. 3
controllers, 3 network nodes, and 2 compute nodes. Using dhcpdump I can see
DHCP requests being made, but no acknowledgement afterwards. If I console
into the server and assign the public IP statically, that IP then works.
It’s only at boot (or reboot), that the instance can’t grab the IP.

Any help would be appreciated, I’ve been banging my head over this for a
while.

Thanks,

Matt C


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
asked Oct 24, 2017 in openstack-operators by mattczajka_at_gmail. (200 points)  

4 Responses

0 votes

On Thu, Oct 19, 2017 at 5:22 PM, Matthew Czajka mattczajka@gmail.com wrote:
Hi All,

I’m running into an issue with a new Ocata deployment. I’m unable to boot an
instance directly from the provider network. Instance logs show that it
keeps repeating “failed to raise interfaces” so it doesn’t get an IP
assigned, and then can’t reach the metatdata servers.

Booting an instance from a self-service network works fine, can ping other
internal IPs within the vlan, and can assign and use a floating IP.

Deployment is running linux bridge, with a flat network for provider. 3
controllers, 3 network nodes, and 2 compute nodes. Using dhcpdump I can see
DHCP requests being made, but no acknowledgement afterwards. If I console
into the server and assign the public IP statically, that IP then works.
It’s only at boot (or reboot), that the instance can’t grab the IP.

Any help would be appreciated, I’ve been banging my head over this for a
while.

Hey Matt :),

To get metadata there has to be a dhcp server or router configured, or
use config drive. Presumably there is no dhcp service running via
neutron in your situation, so it can't get an IP nor the metadata
route injected. You could test with a config drive if you wanted to
verify, or don't want dhcp service. Sometimes organizations refuse
dhcp services on networks, but I'm guessing that is not the case in
this situation.

Basically I expect there is no dhcp service. Double check the network has dhcp?

Thanks,
Curtis.

Thanks,

Matt C


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Blog: serverascode.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 20, 2017 by Curtis (4,180 points)   2 3
0 votes

Oh hey there Curtis!

Basically I expect there is no dhcp service. Double check the network has
dhcp?

There's DHCP running on the net nodes, but the dhcp logs don't show any
errors or warnings

In the DHCP logs, the only difference I can see is when booting from
provider it says its building the host file in /var/lib/neutron/dhcp/ while
booting from a self-service network, it says building in /var/lib/neutron/.

I have enableisolatedmetadata = True in the dhcp config. But I noticed
that the dnsmasq logs dont show a DHCPDISCOVER or OFFER for the provider IP
I try to boot with. It shows for a self-service network.

You could test with a config drive if you wanted to verify, or don't want
dhcp service.

I don't want to use config drive since ceph is the backend, it might mess
up instance migrations I believe?

Thanks,
Matt

On Fri, Oct 20, 2017 at 2:09 PM, Curtis serverascode@gmail.com wrote:

On Thu, Oct 19, 2017 at 5:22 PM, Matthew Czajka mattczajka@gmail.com
wrote:

Hi All,

I’m running into an issue with a new Ocata deployment. I’m unable to
boot an
instance directly from the provider network. Instance logs show that it
keeps repeating “failed to raise interfaces” so it doesn’t get an IP
assigned, and then can’t reach the metatdata servers.

Booting an instance from a self-service network works fine, can ping
other
internal IPs within the vlan, and can assign and use a floating IP.

Deployment is running linux bridge, with a flat network for provider. 3
controllers, 3 network nodes, and 2 compute nodes. Using dhcpdump I can
see
DHCP requests being made, but no acknowledgement afterwards. If I console
into the server and assign the public IP statically, that IP then works.
It’s only at boot (or reboot), that the instance can’t grab the IP.

Any help would be appreciated, I’ve been banging my head over this for a
while.

Hey Matt :),

To get metadata there has to be a dhcp server or router configured, or
use config drive. Presumably there is no dhcp service running via
neutron in your situation, so it can't get an IP nor the metadata
route injected. You could test with a config drive if you wanted to
verify, or don't want dhcp service. Sometimes organizations refuse
dhcp services on networks, but I'm guessing that is not the case in
this situation.

Basically I expect there is no dhcp service. Double check the network has
dhcp?

Thanks,
Curtis.

Thanks,

Matt C


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Blog: serverascode.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 24, 2017 by mattczajka_at_gmail. (200 points)  
0 votes

On Tue, Oct 24, 2017 at 1:58 PM, Matthew Czajka mattczajka@gmail.com wrote:
Oh hey there Curtis!

Basically I expect there is no dhcp service. Double check the network has
dhcp?

There's DHCP running on the net nodes, but the dhcp logs don't show any
errors or warnings

Sorry I missed this...

But is the provider network setup to provide dhcp? If it's not
supposed to, then what we will often do is give an instance two
networks, one for "management" that can use the metadata system and
one that is on the provider network which may not have dhcp support
from openstack.

Thanks,
Curtis.

In the DHCP logs, the only difference I can see is when booting from
provider it says its building the host file in /var/lib/neutron/dhcp/ while
booting from a self-service network, it says building in /var/lib/neutron/.

I have enableisolatedmetadata = True in the dhcp config. But I noticed
that the dnsmasq logs dont show a DHCPDISCOVER or OFFER for the provider IP
I try to boot with. It shows for a self-service network.

You could test with a config drive if you wanted to verify, or don't want
dhcp service.

I don't want to use config drive since ceph is the backend, it might mess up
instance migrations I believe?

Thanks,
Matt

On Fri, Oct 20, 2017 at 2:09 PM, Curtis serverascode@gmail.com wrote:

On Thu, Oct 19, 2017 at 5:22 PM, Matthew Czajka mattczajka@gmail.com
wrote:

Hi All,

I’m running into an issue with a new Ocata deployment. I’m unable to
boot an
instance directly from the provider network. Instance logs show that it
keeps repeating “failed to raise interfaces” so it doesn’t get an IP
assigned, and then can’t reach the metatdata servers.

Booting an instance from a self-service network works fine, can ping
other
internal IPs within the vlan, and can assign and use a floating IP.

Deployment is running linux bridge, with a flat network for provider. 3
controllers, 3 network nodes, and 2 compute nodes. Using dhcpdump I can
see
DHCP requests being made, but no acknowledgement afterwards. If I
console
into the server and assign the public IP statically, that IP then works.
It’s only at boot (or reboot), that the instance can’t grab the IP.

Any help would be appreciated, I’ve been banging my head over this for a
while.

Hey Matt :),

To get metadata there has to be a dhcp server or router configured, or
use config drive. Presumably there is no dhcp service running via
neutron in your situation, so it can't get an IP nor the metadata
route injected. You could test with a config drive if you wanted to
verify, or don't want dhcp service. Sometimes organizations refuse
dhcp services on networks, but I'm guessing that is not the case in
this situation.

Basically I expect there is no dhcp service. Double check the network has
dhcp?

Thanks,
Curtis.

Thanks,

Matt C


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Blog: serverascode.com

--
Blog: serverascode.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 27, 2017 by Curtis (4,180 points)   2 3
0 votes

Ideally I'd like the provider network to handle dhcp by itself without an
additional network for the instance. I would have thought
that enableisolatedmetadata=true would allow any instances provisioned on
the provider network access to the dhcp server. Is there an additional
setting to allow a flat provider network to make use of the dhcp (dnsmasq)
agents on the network nodes?

Thanks,
Matt

On Fri, Oct 27, 2017 at 8:55 AM, Curtis serverascode@gmail.com wrote:

On Tue, Oct 24, 2017 at 1:58 PM, Matthew Czajka mattczajka@gmail.com
wrote:

Oh hey there Curtis!

Basically I expect there is no dhcp service. Double check the network
has
dhcp?

There's DHCP running on the net nodes, but the dhcp logs don't show any
errors or warnings

Sorry I missed this...

But is the provider network setup to provide dhcp? If it's not
supposed to, then what we will often do is give an instance two
networks, one for "management" that can use the metadata system and
one that is on the provider network which may not have dhcp support
from openstack.

Thanks,
Curtis.

In the DHCP logs, the only difference I can see is when booting from
provider it says its building the host file in /var/lib/neutron/dhcp/
while
booting from a self-service network, it says building in
/var/lib/neutron/.

I have enableisolatedmetadata = True in the dhcp config. But I noticed
that the dnsmasq logs dont show a DHCPDISCOVER or OFFER for the provider
IP
I try to boot with. It shows for a self-service network.

You could test with a config drive if you wanted to verify, or don't
want
dhcp service.

I don't want to use config drive since ceph is the backend, it might
mess up
instance migrations I believe?

Thanks,
Matt

On Fri, Oct 20, 2017 at 2:09 PM, Curtis serverascode@gmail.com wrote:

On Thu, Oct 19, 2017 at 5:22 PM, Matthew Czajka mattczajka@gmail.com
wrote:

Hi All,

I’m running into an issue with a new Ocata deployment. I’m unable to
boot an
instance directly from the provider network. Instance logs show that
it
keeps repeating “failed to raise interfaces” so it doesn’t get an IP
assigned, and then can’t reach the metatdata servers.

Booting an instance from a self-service network works fine, can ping
other
internal IPs within the vlan, and can assign and use a floating IP.

Deployment is running linux bridge, with a flat network for provider.
3
controllers, 3 network nodes, and 2 compute nodes. Using dhcpdump I
can
see
DHCP requests being made, but no acknowledgement afterwards. If I
console
into the server and assign the public IP statically, that IP then
works.
It’s only at boot (or reboot), that the instance can’t grab the IP.

Any help would be appreciated, I’ve been banging my head over this
for a
while.

Hey Matt :),

To get metadata there has to be a dhcp server or router configured, or
use config drive. Presumably there is no dhcp service running via
neutron in your situation, so it can't get an IP nor the metadata
route injected. You could test with a config drive if you wanted to
verify, or don't want dhcp service. Sometimes organizations refuse
dhcp services on networks, but I'm guessing that is not the case in
this situation.

Basically I expect there is no dhcp service. Double check the network
has
dhcp?

Thanks,
Curtis.

Thanks,

Matt C


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/
openstack-operators

--
Blog: serverascode.com

--
Blog: serverascode.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 31, 2017 by mattczajka_at_gmail. (200 points)  
...