settingsLogin | Registersettings

[Openstack] How to utilize Neutron independently with veths

0 votes

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of it. What I'm seeking is some advice about how to utilize Neutron independently, to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller node and compute nodes. I guess the following steps are required to test a virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a port for one end of the veth pair (passing veth's mac address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't know the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

asked May 24, 2017 in openstack by duhongwei (460 points)  

16 Responses

0 votes

Hi,

Please take a look at Kuryr https://wiki.openstack.org/wiki/Kuryr. Maybe
this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of it.
What I'm seeking is some advice about how to utilize Neutron independently,
to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller node
and compute nodes. I guess the following steps are required to test a
virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a *port *for one end of the veth pair (passing veth's mac
address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't know
the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 9, 2017 by Vallachorum_Tyranoru (140 points)  
0 votes

Thanks Dan!

I have to write a customized CNI plugin for our product, so it's better if I know more operation details about how to interact with Neutron manually (consider myself as Nova). Therefore Kuryr is not my best option right now, but it's cool!

Regards,
Dastan

------------------ Original ------------------
From: "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Date: Tue, May 9, 2017 04:08 PM
To: "duhongwei"duhongwei@qiniu.com; "openstack"openstack@lists.openstack.org;

Subject: Re: [Openstack] How to utilize Neutron independently with veths

Hi,

Please take a look at Kuryr. Maybe this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of it. What I'm seeking is some advice about how to utilize Neutron independently, to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller node and compute nodes. I guess the following steps are required to test a virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a port for one end of the veth pair (passing veth's mac address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't know the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 9, 2017 by duhongwei (460 points)  
0 votes

You want to look in neutron.agent.linux.interface to see how things are
plugged into OVS. That's the module used by the L3 agent to plug into
OVS/linux bridge/etc.

There is a well defined interface name format corresponding to the port ID
and the port ID, Mac address, and a couple of other things I can't recall
are set in ovsdb to help the agent identify the port as something it should
care about.

On May 9, 2017 04:49, "duhongwei" duhongwei@qiniu.com wrote:

Thanks Dan!

I have to write a customized CNI plugin for our product, so it's better if
I know more operation details about how to interact with Neutron manually
(consider myself as Nova). Therefore Kuryr is not my best option right now,
but it's cool!

Regards,
Dastan

------------------ Original ------------------
*From: * "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
*Date: * Tue, May 9, 2017 04:08 PM
*To: * "duhongwei"duhongwei@qiniu.com; "openstack"<openstack@lists.
openstack.org>;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

Hi,

Please take a look at Kuryr https://wiki.openstack.org/wiki/Kuryr.
Maybe this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of it.
What I'm seeking is some advice about how to utilize Neutron independently,
to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller
node and compute nodes. I guess the following steps are required to test a
virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a *port *for one end of the veth pair (passing veth's mac
address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't know
the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/
openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/
openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/
openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/
openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 12, 2017 by kevin_at_benton.pub (15,600 points)   2 3 5
0 votes

Thanks Kevin!

I'll dig into neutron.agent.linux.interface to see how it works. Before that, would you give me any previews about what steps should be taken to add a veth to a existed Neutron network?

Furthermore, is it Neutron who add a veth to ovs bridge or is it the Neutron caller? (such as Nova)

Who's adding flows to ovs bridge? Neutron or caller?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Fri, May 12, 2017 10:45 AM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

You want to look in neutron.agent.linux.interface to see how things are plugged into OVS. That's the module used by the L3 agent to plug into OVS/linux bridge/etc.

There is a well defined interface name format corresponding to the port ID and the port ID, Mac address, and a couple of other things I can't recall are set in ovsdb to help the agent identify the port as something it should care about.

On May 9, 2017 04:49, "duhongwei" duhongwei@qiniu.com wrote:

Thanks Dan!

I have to write a customized CNI plugin for our product, so it's better if I know more operation details about how to interact with Neutron manually (consider myself as Nova). Therefore Kuryr is not my best option right now, but it's cool!

Regards,
Dastan

------------------ Original ------------------
From: "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Date: Tue, May 9, 2017 04:08 PM
To: "duhongwei"duhongwei@qiniu.com; "openstack"openstack@lists.openstack.org;

Subject: Re: [Openstack] How to utilize Neutron independently with veths

Hi,

Please take a look at Kuryr. Maybe this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of it. What I'm seeking is some advice about how to utilize Neutron independently, to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller node and compute nodes. I guess the following steps are required to test a virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a port for one end of the veth pair (passing veth's mac address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't know the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 12, 2017 by duhongwei (460 points)  
0 votes

Nova is only responsible for creating the interface and plugging it into
the OVS bridge. It's the neutron agent (or alternative neutron backend like
OVN) responsible for setting up all of the flows.

Here is a hacky script that I had used to create and delete a bunch of
ports like Nova would that you can probably start with:
http://paste.openstack.org/show/609478/

On Fri, May 12, 2017 at 4:25 AM, duhongwei duhongwei@qiniu.com wrote:

Thanks Kevin!

I'll dig into neutron.agent.linux.interface to see how it works. Before
that, would you give me any previews about what steps should be taken to
add a veth to a existed Neutron network?

Furthermore, is it Neutron who add a veth to ovs bridge or is it the
Neutron caller? (such as Nova)

Who's adding flows to ovs bridge? Neutron or caller?

Regards,
Dastan

------------------ Original ------------------
*From: * "Kevin Benton"kevin@benton.pub;
*Date: * Fri, May 12, 2017 10:45 AM
*To: * "duhongwei"duhongwei@qiniu.com;
*Cc: * "openstack"openstack@lists.openstack.org; "Vallachorum
Tyranorum"ardeleandanflorin@gmail.com;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

You want to look in neutron.agent.linux.interface to see how things are
plugged into OVS. That's the module used by the L3 agent to plug into
OVS/linux bridge/etc.

There is a well defined interface name format corresponding to the port ID
and the port ID, Mac address, and a couple of other things I can't recall
are set in ovsdb to help the agent identify the port as something it should
care about.

On May 9, 2017 04:49, "duhongwei" duhongwei@qiniu.com wrote:

Thanks Dan!

I have to write a customized CNI plugin for our product, so it's better
if I know more operation details about how to interact with Neutron
manually (consider myself as Nova). Therefore Kuryr is not my best option
right now, but it's cool!

Regards,
Dastan

------------------ Original ------------------
*From: * "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
*Date: * Tue, May 9, 2017 04:08 PM
*To: * "duhongwei"duhongwei@qiniu.com; "openstack"<openstack@lists.op
enstack.org>;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

Hi,

Please take a look at Kuryr https://wiki.openstack.org/wiki/Kuryr.
Maybe this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of it.
What I'm seeking is some advice about how to utilize Neutron independently,
to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller
node and compute nodes. I guess the following steps are required to test a
virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a *port *for one end of the veth pair (passing veth's mac
address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't know
the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 12, 2017 by kevin_at_benton.pub (15,600 points)   2 3 5
0 votes

This script seems easy and cool!

So first we have to create a logical neutron port, then create qbr, qvo and qvb, and plug qvb into qbr, finally plug qvo into ovs br-int. After all these, we create veth/tap (as vm/containers vNIC) and plugin it into qbr then we're able to talk with other vms/containers on the same network through veth/tap, am I understanding it right?

Questions,

1) isn't it necessary that veth/tap's mac address same as neutron port's mac address?
2) after we plug qvo into ovs br-int, neutron just automatically add flows into ovs bridge?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Sat, May 13, 2017 07:46 AM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

Nova is only responsible for creating the interface and plugging it into the OVS bridge. It's the neutron agent (or alternative neutron backend like OVN) responsible for setting up all of the flows.

Here is a hacky script that I had used to create and delete a bunch of ports like Nova would that you can probably start with: http://paste.openstack.org/show/609478/

On Fri, May 12, 2017 at 4:25 AM, duhongwei duhongwei@qiniu.com wrote:

Thanks Kevin!

I'll dig into neutron.agent.linux.interface to see how it works. Before that, would you give me any previews about what steps should be taken to add a veth to a existed Neutron network?

Furthermore, is it Neutron who add a veth to ovs bridge or is it the Neutron caller? (such as Nova)

Who's adding flows to ovs bridge? Neutron or caller?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Fri, May 12, 2017 10:45 AM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

You want to look in neutron.agent.linux.interface to see how things are plugged into OVS. That's the module used by the L3 agent to plug into OVS/linux bridge/etc.

There is a well defined interface name format corresponding to the port ID and the port ID, Mac address, and a couple of other things I can't recall are set in ovsdb to help the agent identify the port as something it should care about.

On May 9, 2017 04:49, "duhongwei" duhongwei@qiniu.com wrote:

Thanks Dan!

I have to write a customized CNI plugin for our product, so it's better if I know more operation details about how to interact with Neutron manually (consider myself as Nova). Therefore Kuryr is not my best option right now, but it's cool!

Regards,
Dastan

------------------ Original ------------------
From: "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Date: Tue, May 9, 2017 04:08 PM
To: "duhongwei"duhongwei@qiniu.com; "openstack"openstack@lists.openstack.org;

Subject: Re: [Openstack] How to utilize Neutron independently with veths

Hi,

Please take a look at Kuryr. Maybe this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of it. What I'm seeking is some advice about how to utilize Neutron independently, to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller node and compute nodes. I guess the following steps are required to test a virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a port for one end of the veth pair (passing veth's mac address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't know the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 19, 2017 by duhongwei (460 points)  
0 votes

After all these, we create veth/tap (as vm/containers vNIC) and plugin
it into qbr then we're able to talk with other vms/containers on the same
network through veth/tap, am I understanding it right?

Yes, this last step of creating a veth/tap is missing from my script
because I didn't need actual dataplane communication for the tests I was
doing.

1) isn't it necessary that veth/tap's mac address same as neutron port's
mac address?

Yeah, if you attach something to qbr to behave like the VM interface, you
will need it to be using the mac address of the neutron port, or else the
neutron anti-spoofing rules will prevent it from communicating.

2) after we plug qvo into ovs br-int, neutron just automatically add
flows into ovs bridge?

Yes, the agent will receive to the new port event from ovs, retrieve port
details from the server and then setup the flows.

On Fri, May 19, 2017 at 12:09 AM, duhongwei duhongwei@qiniu.com wrote:

This script seems easy and cool!

So first we have to create a logical neutron port, then create qbr,
qvo and qvb, and plug qvb into qbr, finally plug qvo into ovs
br-int. After all these, we create veth/tap (as vm/containers vNIC)
and plugin it into qbr then we're able to talk with other
vms/containers on the same network through veth/tap, am I understanding
it right?

Questions,

1) isn't it necessary that veth/tap's mac address same as neutron port's
mac address?
2) after we plug qvo into ovs br-int, neutron just automatically add
flows into ovs bridge?

Regards,
Dastan

------------------ Original ------------------
*From: * "Kevin Benton"kevin@benton.pub;
*Date: * Sat, May 13, 2017 07:46 AM
*To: * "duhongwei"duhongwei@qiniu.com;
*Cc: * "openstack"openstack@lists.openstack.org; "Vallachorum
Tyranorum"ardeleandanflorin@gmail.com;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

Nova is only responsible for creating the interface and plugging it into
the OVS bridge. It's the neutron agent (or alternative neutron backend like
OVN) responsible for setting up all of the flows.

Here is a hacky script that I had used to create and delete a bunch of
ports like Nova would that you can probably start with:
http://paste.openstack.org/show/609478/

On Fri, May 12, 2017 at 4:25 AM, duhongwei duhongwei@qiniu.com wrote:

Thanks Kevin!

I'll dig into neutron.agent.linux.interface to see how it works. Before
that, would you give me any previews about what steps should be taken to
add a veth to a existed Neutron network?

Furthermore, is it Neutron who add a veth to ovs bridge or is it the
Neutron caller? (such as Nova)

Who's adding flows to ovs bridge? Neutron or caller?

Regards,
Dastan

------------------ Original ------------------
*From: * "Kevin Benton"kevin@benton.pub;
*Date: * Fri, May 12, 2017 10:45 AM
*To: * "duhongwei"duhongwei@qiniu.com;
*Cc: * "openstack"openstack@lists.openstack.org; "Vallachorum
Tyranorum"ardeleandanflorin@gmail.com;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

You want to look in neutron.agent.linux.interface to see how things are
plugged into OVS. That's the module used by the L3 agent to plug into
OVS/linux bridge/etc.

There is a well defined interface name format corresponding to the port
ID and the port ID, Mac address, and a couple of other things I can't
recall are set in ovsdb to help the agent identify the port as something it
should care about.

On May 9, 2017 04:49, "duhongwei" duhongwei@qiniu.com wrote:

Thanks Dan!

I have to write a customized CNI plugin for our product, so it's better
if I know more operation details about how to interact with Neutron
manually (consider myself as Nova). Therefore Kuryr is not my best option
right now, but it's cool!

Regards,
Dastan

------------------ Original ------------------
*From: * "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
*Date: * Tue, May 9, 2017 04:08 PM
*To: * "duhongwei"duhongwei@qiniu.com; "openstack"<openstack@lists.op
enstack.org>;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

Hi,

Please take a look at Kuryr https://wiki.openstack.org/wiki/Kuryr.
Maybe this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of
it. What I'm seeking is some advice about how to utilize Neutron
independently, to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller
node and compute nodes. I guess the following steps are required to test a
virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a *port *for one end of the veth pair (passing veth's mac
address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't
know the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 19, 2017 by kevin_at_benton.pub (15,600 points)   2 3 5
0 votes

Thanks for your patient, Kevin.

So qvo could be any veth whose mac address doesn't matter, but veth/tap must have exact the same mac address as port, otherwise it will be anti-spoofed.

qvo's attributes (external-ids) tell neutron which logical port qvo is connecting, so neutron knows how to add flows to ovs br-int and br-tun.

Am I correct?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Sat, May 20, 2017 03:26 AM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

After all these, we create veth/tap (as vm/containers vNIC) and plugin it into qbr then we're able to talk with other vms/containers on the same network through veth/tap, am I understanding it right?

Yes, this last step of creating a veth/tap is missing from my script because I didn't need actual dataplane communication for the tests I was doing.

1) isn't it necessary that veth/tap's mac address same as neutron port's mac address?

Yeah, if you attach something to qbr to behave like the VM interface, you will need it to be using the mac address of the neutron port, or else the neutron anti-spoofing rules will prevent it from communicating.

2) after we plug qvo into ovs br-int, neutron just automatically add flows into ovs bridge?

Yes, the agent will receive to the new port event from ovs, retrieve port details from the server and then setup the flows.

On Fri, May 19, 2017 at 12:09 AM, duhongwei duhongwei@qiniu.com wrote:

This script seems easy and cool!

So first we have to create a logical neutron port, then create qbr, qvo and qvb, and plug qvb into qbr, finally plug qvo into ovs br-int. After all these, we create veth/tap (as vm/containers vNIC) and plugin it into qbr then we're able to talk with other vms/containers on the same network through veth/tap, am I understanding it right?

Questions,

1) isn't it necessary that veth/tap's mac address same as neutron port's mac address?
2) after we plug qvo into ovs br-int, neutron just automatically add flows into ovs bridge?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Sat, May 13, 2017 07:46 AM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

Nova is only responsible for creating the interface and plugging it into the OVS bridge. It's the neutron agent (or alternative neutron backend like OVN) responsible for setting up all of the flows.

Here is a hacky script that I had used to create and delete a bunch of ports like Nova would that you can probably start with: http://paste.openstack.org/show/609478/

On Fri, May 12, 2017 at 4:25 AM, duhongwei duhongwei@qiniu.com wrote:

Thanks Kevin!

I'll dig into neutron.agent.linux.interface to see how it works. Before that, would you give me any previews about what steps should be taken to add a veth to a existed Neutron network?

Furthermore, is it Neutron who add a veth to ovs bridge or is it the Neutron caller? (such as Nova)

Who's adding flows to ovs bridge? Neutron or caller?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Fri, May 12, 2017 10:45 AM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

You want to look in neutron.agent.linux.interface to see how things are plugged into OVS. That's the module used by the L3 agent to plug into OVS/linux bridge/etc.

There is a well defined interface name format corresponding to the port ID and the port ID, Mac address, and a couple of other things I can't recall are set in ovsdb to help the agent identify the port as something it should care about.

On May 9, 2017 04:49, "duhongwei" duhongwei@qiniu.com wrote:

Thanks Dan!

I have to write a customized CNI plugin for our product, so it's better if I know more operation details about how to interact with Neutron manually (consider myself as Nova). Therefore Kuryr is not my best option right now, but it's cool!

Regards,
Dastan

------------------ Original ------------------
From: "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Date: Tue, May 9, 2017 04:08 PM
To: "duhongwei"duhongwei@qiniu.com; "openstack"openstack@lists.openstack.org;

Subject: Re: [Openstack] How to utilize Neutron independently with veths

Hi,

Please take a look at Kuryr. Maybe this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of it. What I'm seeking is some advice about how to utilize Neutron independently, to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller node and compute nodes. I guess the following steps are required to test a virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a port for one end of the veth pair (passing veth's mac address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't know the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 22, 2017 by duhongwei (460 points)  
0 votes

Yes, the only thing that needs to use the correct MAC is whatever is
actually sending traffic.

On May 21, 2017 22:06, "duhongwei" duhongwei@qiniu.com wrote:

Thanks for your patient, Kevin.

So qvo *could be any veth whose mac address doesn't matter, but *veth/tap
*must have exact the same mac address as *port
, otherwise it will be
anti-spoofed.

qvo's attributes (external-ids) tell neutron which logical port qvo
is connecting, so neutron knows how to add flows to ovs br-int *and
*br-tun
.

Am I correct?

Regards,
Dastan

------------------ Original ------------------
*From: * "Kevin Benton"kevin@benton.pub;
*Date: * Sat, May 20, 2017 03:26 AM
*To: * "duhongwei"duhongwei@qiniu.com;
*Cc: * "openstack"openstack@lists.openstack.org; "Vallachorum
Tyranorum"ardeleandanflorin@gmail.com;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

After all these, we create veth/tap (as vm/containers vNIC) and plugin
it into qbr then we're able to talk with other vms/containers on the
same network through veth/tap, am I understanding it right?

Yes, this last step of creating a veth/tap is missing from my script
because I didn't need actual dataplane communication for the tests I was
doing.

1) isn't it necessary that veth/tap's mac address same as neutron
port's mac address?

Yeah, if you attach something to qbr to behave like the VM interface, you
will need it to be using the mac address of the neutron port, or else the
neutron anti-spoofing rules will prevent it from communicating.

2) after we plug qvo into ovs br-int, neutron just automatically add
flows into ovs bridge?

Yes, the agent will receive to the new port event from ovs, retrieve port
details from the server and then setup the flows.

On Fri, May 19, 2017 at 12:09 AM, duhongwei duhongwei@qiniu.com wrote:

This script seems easy and cool!

So first we have to create a logical neutron port, then create qbr,
qvo and qvb, and plug qvb into qbr, finally plug qvo into ovs
br-int. After all these, we create veth/tap (as vm/containers vNIC)
and plugin it into qbr then we're able to talk with other
vms/containers on the same network through veth/tap, am I
understanding it right?

Questions,

1) isn't it necessary that veth/tap's mac address same as neutron
port's mac address?
2) after we plug qvo into ovs br-int, neutron just automatically add
flows into ovs bridge?

Regards,
Dastan

------------------ Original ------------------
*From: * "Kevin Benton"kevin@benton.pub;
*Date: * Sat, May 13, 2017 07:46 AM
*To: * "duhongwei"duhongwei@qiniu.com;
*Cc: * "openstack"openstack@lists.openstack.org; "Vallachorum
Tyranorum"ardeleandanflorin@gmail.com;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

Nova is only responsible for creating the interface and plugging it into
the OVS bridge. It's the neutron agent (or alternative neutron backend like
OVN) responsible for setting up all of the flows.

Here is a hacky script that I had used to create and delete a bunch of
ports like Nova would that you can probably start with:
http://paste.openstack.org/show/609478/

On Fri, May 12, 2017 at 4:25 AM, duhongwei duhongwei@qiniu.com wrote:

Thanks Kevin!

I'll dig into neutron.agent.linux.interface to see how it works. Before
that, would you give me any previews about what steps should be taken to
add a veth to a existed Neutron network?

Furthermore, is it Neutron who add a veth to ovs bridge or is it the
Neutron caller? (such as Nova)

Who's adding flows to ovs bridge? Neutron or caller?

Regards,
Dastan

------------------ Original ------------------
*From: * "Kevin Benton"kevin@benton.pub;
*Date: * Fri, May 12, 2017 10:45 AM
*To: * "duhongwei"duhongwei@qiniu.com;
*Cc: * "openstack"openstack@lists.openstack.org; "Vallachorum
Tyranorum"ardeleandanflorin@gmail.com;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

You want to look in neutron.agent.linux.interface to see how things are
plugged into OVS. That's the module used by the L3 agent to plug into
OVS/linux bridge/etc.

There is a well defined interface name format corresponding to the port
ID and the port ID, Mac address, and a couple of other things I can't
recall are set in ovsdb to help the agent identify the port as something it
should care about.

On May 9, 2017 04:49, "duhongwei" duhongwei@qiniu.com wrote:

Thanks Dan!

I have to write a customized CNI plugin for our product, so it's better
if I know more operation details about how to interact with Neutron
manually (consider myself as Nova). Therefore Kuryr is not my best option
right now, but it's cool!

Regards,
Dastan

------------------ Original ------------------
*From: * "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
*Date: * Tue, May 9, 2017 04:08 PM
*To: * "duhongwei"duhongwei@qiniu.com; "openstack"<openstack@lists.op
enstack.org>;
*Subject: * Re: [Openstack] How to utilize Neutron independently with
veths

Hi,

Please take a look at Kuryr https://wiki.openstack.org/wiki/Kuryr.
Maybe this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of
it. What I'm seeking is some advice about how to utilize Neutron
independently, to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller
node and compute nodes. I guess the following steps are required to test a
virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a *port *for one end of the veth pair (passing veth's mac
address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't
know the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 22, 2017 by kevin_at_benton.pub (15,600 points)   2 3 5
0 votes

Thanks Kevin! I've made a big step forward!

Till now, I've successfully connect vNIC directly into br-int without qbr, qvo, and qvb. And, it works well.

However, following your scripts (connect vNIC into qbr, then connect qbr into br-int) exposes another problem. In this scenario, qbr won't forward packets from vNIC to br-int (packets seem to be dropped on qbr).

After some troubleshooting, it turns out to be iptables who drops packets on qbr. Reviewing the FORWARD chain in filter table, packets come from vNIC won't match any rule of neutron-filter-top and neutron-openvswi-FORWARD so that the default policy DROP applies.

So, after setting up all these qbr, qvo, qvb, vNIC, it seems there're still some iptables rules missed. Question is,

Who's adding this iptables rules? (Nova or Neutron?) How can I make it happen?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Mon, May 22, 2017 10:47 PM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

Yes, the only thing that needs to use the correct MAC is whatever is actually sending traffic.

On May 21, 2017 22:06, "duhongwei" duhongwei@qiniu.com wrote:

Thanks for your patient, Kevin.

So qvo could be any veth whose mac address doesn't matter, but veth/tap must have exact the same mac address as port, otherwise it will be anti-spoofed.

qvo's attributes (external-ids) tell neutron which logical port qvo is connecting, so neutron knows how to add flows to ovs br-int and br-tun.

Am I correct?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Sat, May 20, 2017 03:26 AM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

After all these, we create veth/tap (as vm/containers vNIC) and plugin it into qbr then we're able to talk with other vms/containers on the same network through veth/tap, am I understanding it right?

Yes, this last step of creating a veth/tap is missing from my script because I didn't need actual dataplane communication for the tests I was doing.

1) isn't it necessary that veth/tap's mac address same as neutron port's mac address?

Yeah, if you attach something to qbr to behave like the VM interface, you will need it to be using the mac address of the neutron port, or else the neutron anti-spoofing rules will prevent it from communicating.

2) after we plug qvo into ovs br-int, neutron just automatically add flows into ovs bridge?

Yes, the agent will receive to the new port event from ovs, retrieve port details from the server and then setup the flows.

On Fri, May 19, 2017 at 12:09 AM, duhongwei duhongwei@qiniu.com wrote:

This script seems easy and cool!

So first we have to create a logical neutron port, then create qbr, qvo and qvb, and plug qvb into qbr, finally plug qvo into ovs br-int. After all these, we create veth/tap (as vm/containers vNIC) and plugin it into qbr then we're able to talk with other vms/containers on the same network through veth/tap, am I understanding it right?

Questions,

1) isn't it necessary that veth/tap's mac address same as neutron port's mac address?
2) after we plug qvo into ovs br-int, neutron just automatically add flows into ovs bridge?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Sat, May 13, 2017 07:46 AM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

Nova is only responsible for creating the interface and plugging it into the OVS bridge. It's the neutron agent (or alternative neutron backend like OVN) responsible for setting up all of the flows.

Here is a hacky script that I had used to create and delete a bunch of ports like Nova would that you can probably start with: http://paste.openstack.org/show/609478/

On Fri, May 12, 2017 at 4:25 AM, duhongwei duhongwei@qiniu.com wrote:

Thanks Kevin!

I'll dig into neutron.agent.linux.interface to see how it works. Before that, would you give me any previews about what steps should be taken to add a veth to a existed Neutron network?

Furthermore, is it Neutron who add a veth to ovs bridge or is it the Neutron caller? (such as Nova)

Who's adding flows to ovs bridge? Neutron or caller?

Regards,
Dastan

------------------ Original ------------------
From: "Kevin Benton"kevin@benton.pub;
Date: Fri, May 12, 2017 10:45 AM
To: "duhongwei"duhongwei@qiniu.com;
Cc: "openstack"openstack@lists.openstack.org; "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Subject: Re: [Openstack] How to utilize Neutron independently with veths

You want to look in neutron.agent.linux.interface to see how things are plugged into OVS. That's the module used by the L3 agent to plug into OVS/linux bridge/etc.

There is a well defined interface name format corresponding to the port ID and the port ID, Mac address, and a couple of other things I can't recall are set in ovsdb to help the agent identify the port as something it should care about.

On May 9, 2017 04:49, "duhongwei" duhongwei@qiniu.com wrote:

Thanks Dan!

I have to write a customized CNI plugin for our product, so it's better if I know more operation details about how to interact with Neutron manually (consider myself as Nova). Therefore Kuryr is not my best option right now, but it's cool!

Regards,
Dastan

------------------ Original ------------------
From: "Vallachorum Tyranorum"ardeleandanflorin@gmail.com;
Date: Tue, May 9, 2017 04:08 PM
To: "duhongwei"duhongwei@qiniu.com; "openstack"openstack@lists.openstack.org;

Subject: Re: [Openstack] How to utilize Neutron independently with veths

Hi,

Please take a look at Kuryr. Maybe this is what you are looking for.

Dan.

On Tue, May 9, 2017 at 10:17 AM duhongwei duhongwei@qiniu.com wrote:

Hi everyone,

I'm new to OpenStack and currently interested in the Neutron part of it. What I'm seeking is some advice about how to utilize Neutron independently, to build a virtual network, for Docker containers maybe.

Suppose I've already got Neutron and Keystone installed on controller node and compute nodes. I guess the following steps are required to test a virtual network.

1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a port for one end of the veth pair (passing veth's mac address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another

The above is my general idea, don't know if it is correct and don't know the operation details either.
Expecting your suggestions, any links are appreciated.

Regards,
Dastan


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded May 23, 2017 by duhongwei (460 points)  
...