settingsLogin | Registersettings

[openstack-dev] [ironic] [nova] traits discussion call

0 votes

Hi all!

I'd like to invite you to the discussion of the way to implement traits in
ironic and the ironic virt driver. Please vote for the time at
https://doodle.com/poll/ts43k98kkvniv8uz. Please vote by EOD tomorrow.

Note that it's going to be a technical discussion - please make sure you
understand what traits are and why ironic cares about them. See below for more
context.

We'll probably use my bluejeans account, as it works without plugins in modern
browsers. I'll post a meeting ID when we pick the date.

On 10/23/2017 04:09 PM, Eric Fried wrote:

We discussed this a little bit further in IRC [1]. We're all in
agreement, but it's worth being precise on a couple of points:

  • We're distinguishing between a "feature" and the "trait" that
    represents it in placement. For the sake of this discussion, a
    "feature" can (maybe) be switched on or off, but a "trait" can either be
    present or absent on a RP.
  • It matters who can turn a feature on/off.

    • If it can be done by virt at spawn time, then it makes sense to have
      the trait on the RP, and you can switch the feature on/off via a
      separate extra_spec.
    • But if it's e.g. an admin action, and spawn has no control, then the
      trait needs to be added whenever the feature is on, and removed
      whenever the feature is off.

[1]
http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13

On 10/23/2017 08:15 AM, Sylvain Bauza wrote:

On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried <openstack@fried.cc
openstack@fried.cc> wrote:

 I agree with Sean.  In general terms:

 * A resource provider should be marked with a trait if that feature
   * Can be turned on or off (whether it's currently on or not); or
   * Is always on and can't ever be turned off.

No, traits are not boolean. If a resource provider stops providing a
capability, then the existing related trait should just be removed,
that's it.
If you see a trait, that's just means that the related capability for
the Resource Provider is supported, that's it too.

MHO.

-Sylvain

 * A consumer wanting that feature present (doesn't matter whether it's
 on or off) should specify it as a required *trait*.
 * A consumer wanting that feature present and turned on should
   * Specify it as a required trait; AND
   * Indicate that it be turned on via some other mechanism (e.g. a
 separate extra_spec).

 I believe this satisfies Dmitry's (Ironic's) needs, but also Jay's drive
 for placement purity.

 Please invite me to the hangout or whatever.

 Thanks,
 Eric

 On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
 >
 >
 >
 >
 > *From:*Jay Pipes [mailto:jaypipes@gmail.com
 <mailto:jaypipes@gmail.com>]
 > *Sent:* Monday, October 23, 2017 12:20 PM
 > *To:* OpenStack Development Mailing List
 <openstack-dev@lists.openstack.org
 <mailto:openstack-dev@lists.openstack.org>>
 > *Subject:* Re: [openstack-dev] [ironic] ironic and traits
 >
 >
 >
 > Writing from my phone... May I ask that before you proceed with any plan
 > that uses traits for state information that we have a hangout or
 > videoconference to discuss this? Unfortunately today and tomorrow I'm
 > not able to do a hangout but I can do one on Wednesday any time of the day.
 >
 >
 >
 > */[Mooney, Sean K] on the uefi boot topic I did bring up at the
 ptg that
 > we wanted to standardizes tratis for “verified boot” /*
 >
 > */that included a trait for uefi secure boot enabled and to
 indicated a
 > hardware root of trust, e.g. intel boot guard or similar/*
 >
 > */we distinctly wanted to be able to tag nova compute hosts with those
 > new traits so we could require that vms that request/*
 >
 > */a host with uefi secure boot enabled and a hardware root of
 trust are
 > scheduled only to those nodes. /*
 >
 > */ /*
 >
 > */There are many other examples that effect both vms and bare
 metal such
 > as, ecc/interleaved memory, cluster on die, /*
 >
 > */l3 cache code and data prioritization, vt-d/vt-c, HPET, Hyper
 > threading, power states … all of these feature may be present on the
 > platform/*
 >
 > */but I also need to know if they are turned on. Ruling out state in
 > traits means all of this logic will eventually get pushed to scheduler
 > filters/*
 >
 > */which will be suboptimal long term as more state is tracked.
 Software
 > defined infrastructure may be the future but hardware defined
 software/*
 >
 > */is sadly the present…/*
 >
 > */ /*
 >
 > */I do however think there should be a sperateion between asking for a
 > host that provides x with a trait and  asking for x to be
 configure via/*
 >
 > */A trait. The trait secure_boot_enabled should never result in the
 > feature being enabled It should just find a host with it on. If
 you want/*
 >
 > */To request it to be turned on you would request a host with
 > secure_boot_capable as a trait and have a flavor extra spec or image
 > property to request/*
 >
 > */Ironic to enabled it.  these are two very different request and
 should
 > not be treated the same. /*
 >
 >
 >
 >
 >
 > Lemme know!
 >
 > -jay
 >
 >
 >
 > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur" <dtantsur@redhat.com <mailto:dtantsur@redhat.com>
 > <mailto:dtantsur@redhat.com <mailto:dtantsur@redhat.com>>> wrote:
 >
 >     Hi Jay!
 >
 >     I appreciate your comments, but I think you're approaching the
 >     problem from purely VM point of view. Things simply don't work the
 >     same way in bare metal, at least not if we want to provide the same
 >     user experience.
 >
 >
 >
 >     On Sun, Oct 22, 2017 at 2:25 PM, Jay Pipes <jaypipes@gmail.com <mailto:jaypipes@gmail.com>
 >     <mailto:jaypipes@gmail.com <mailto:jaypipes@gmail.com>>> wrote:
 >
 >         Sorry for delay, took a week off before starting a new job.
 >         Comments inline.
 >
 >         On 10/16/2017 12:24 PM, Dmitry Tantsur wrote:
 >
 >             Hi all,
 >
 >             I promised John to dump my thoughts on traits to the
 ML, so
 >             here we go :)
 >
 >             I see two roles of traits (or kinds of traits) for
 bare metal:
 >             1. traits that say what the node can do already (e.g. "the
 >             node is
 >             doing UEFI boot")
 >             2. traits that say what the node can be *configured* to do
 >             (e.g. "the node can
 >             boot in UEFI mode")
 >
 >
 >         There's only one role for traits. #2 above. #1 is state
 >         information. Traits are not for state information. Traits are
 >         only for communicating capabilities of a resource provider
 >         (baremetal node).
 >
 >
 >
 >     These are not different, that's what I'm talking about here. No
 >     users care about the difference between "this node was put in UEFI
 >     mode by an operator in advance", "this node was put in UEFI
 mode by
 >     an ironic driver on demand" and "this node is always in UEFI mode,
 >     because it's AARCH64 and it does not have BIOS". These situation
 >     produce the same result (the node is booted in UEFI mode), and
 thus
 >     it's up to ironic to hide this difference.
 >
 >
 >
 >     My suggestion with traits is one way to do it, I'm not sure
 what you
 >     suggest though.
 >
 >
 >
 >
 >         For example, let's say we add the following to the os-traits
 >         library [1]
 >
 >         * STORAGE_RAID_0
 >         * STORAGE_RAID_1
 >         * STORAGE_RAID_5
 >         * STORAGE_RAID_6
 >         * STORAGE_RAID_10
 >
 >         The Ironic administrator would add all RAID-related traits to
 >         the baremetal nodes that had the *capability* of
 supporting that
 >         particular RAID setup [2]
 >
 >         When provisioned, the baremetal node would either have RAID
 >         configured in a certain level or not configured at all.
 >
 >
 >         A very important note: the Placement API and Nova
 scheduler (or
 >         future Ironic scheduler) doesn't care about this. At all.
 I know
 >         it sounds like I'm being callous, but I'm not. Placement and
 >         scheduling doesn't care about the state of things. It only
 cares
 >         about the capabilities of target destinations. That's it.
 >
 >
 >
 >     Yes, because VMs always start with a clean state, and
 hypervisor is
 >     there to ensure that. We don't have this luxury in ironic :) E.g.
 >     our SNMP driver is not even aware of boot modes (or RAID, or BIOS
 >     configuration), which does not mean that a node using it cannot be
 >     in UEFI mode (have a RAID or BIOS pre-configured, etc, etc).
 >
 >
 >
 >
 >
 >             This seems confusing, but it's actually very useful.
 Say, I
 >             have a flavor that
 >             requests UEFI boot via a trait. It will match both the
 nodes
 >             that are already in
 >             UEFI mode, as well as nodes that can be put in UEFI mode.
 >
 >
 >         No :) It will only match nodes that have the UEFI capability.
 >         The set of providers that have the ability to be booted
 via UEFI
 >         is *always* a superset of the set of providers that *have been
 >         booted via UEFI*. Placement and scheduling decisions only care
 >         about that superset -- the providers with a particular
 capability.
 >
 >
 >
 >     Well, no, it will. Again, you're purely basing on the VM idea,
 where
 >     a VM is always *put* in UEFI mode, no matter how the hypervisor
 >     looks like. It is simply not the case for us. You have to care
 what
 >     state the node is, because many drivers cannot change this state.
 >
 >
 >
 >
 >
 >             This idea goes further with deploy templates (new concept
 >             we've been thinking
 >             about). A flavor can request something like CUSTOM_RAID_5,
 >             and it will match the
 >             nodes that already have RAID 5, or, more
 interestingly, the
 >             nodes on which we
 >             can build RAID 5 before deployment. The UEFI example above
 >             can be treated in a
 >             similar way.
 >
 >             This ends up with two sources of knowledge about traits in
 >             ironic:
 >             1. Operators setting something they know about hardware
 >             ("this node is in UEFI
 >             mode"),
 >             2. Ironic drivers reporting something they
 >                2.1. know about hardware ("this node is in UEFI mode" -
 >             again)
 >                2.2. can do about hardware ("I can put this node in
 UEFI
 >             mode")
 >
 >
 >         You're correct that both pieces of information are important.
 >         However, only the "can do about hardware" part is relevant to
 >         Placement and Nova.
 >
 >             For case #1 we are planning on a new CRUD API to set/unset
 >             traits for a node.
 >
 >
 >         I would *strongly* advise against this. Traits are not for
 state
 >         information.
 >
 >         Instead, consider having a DB (or JSON) schema that lists
 state
 >         information in fields that are explicitly for that state
 >         information.
 >
 >         For example, a schema that looks like this:
 >
 >         {
 >           "boot": {
 >             "mode": <one of 'bios' or 'uefi'>,
 >             "params": <dict>
 >           },
 >           "disk": {
 >             "raid": {
 >               "level": <int>,
 >               "controller": <one of 'sw' or 'hw'>,
 >               "driver": <string>,
 >               "params": <dict>
 >             },  ...
 >           },
 >           "network": {
 >             ...
 >           }
 >         }
 >
 >         etc, etc.
 >
 >         Don't use trait strings to represent state information.
 >
 >
 >
 >     I don't see an alternative proposal that will satisfy what we have
 >     to solve.
 >
 >
 >
 >
 >         Best,
 >         -jay
 >
 >             Case #2 is more interesting. We have two options, I think:
 >
 >             a) Operators still set traits on nodes, drivers are simply
 >             validating them. E.g.
 >             an operators sets CUSTOM_RAID_5, and the node's RAID
 >             interface checks if it is
 >             possible to do. The downside is obvious - with a lot of
 >             deploy templates
 >             available it can be a lot of manual work.
 >
 >             b) Drivers report the traits, and they get somehow
 added to
 >             the traits provided
 >             by an operator. Technically, there are sub-cases again:
 >                b.1) The new traits API returns a union of
 >             operator-provided and
 >             driver-provided traits
 >                b.2) The new traits API returns only operator-provided
 >             traits; driver-provided
 >             traits are returned e.g. via a new field
 >             (node.driver_traits). Then nova will
 >             have to merge the lists itself.
 >
 >             My personal favorite is the last option: I'd like a clear
 >             distinction between
 >             different "sources" of traits, but I'd also like to reduce
 >             manual work for
 >             operators.
 >
 >             A valid counter-argument is: what if an operator wants to
 >             override a
 >             driver-provided trait? E.g. a node can do RAID 5, but I
 >             don't want this
 >             particular node to do it for any reason. I'm not sure if
 >             it's a valid case, and
 >             what to do about it.
 >
 >             Let me know what you think.
 >
 >             Dmitry
 >
 >
 >         [1]
 http://git.openstack.org/cgit/openstack/os-traits/tree/
 <http://git.openstack.org/cgit/openstack/os-traits/tree/>
 >         [2] Based on how many attached disks the node had, the
 presence
 >         and abilities of a hardware RAID controller, etc
 >
 >
 >
 >
  __________________________________________________________________________
 >         OpenStack Development Mailing List (not for usage questions)
 >         Unsubscribe:
 >
  OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
 <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
 >
  <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
 <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
 >         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
 >
 >
 >
 >
 >     __________________________________________________________________________
 >     OpenStack Development Mailing List (not for usage questions)
 >     Unsubscribe:
 >     OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
 <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
 >
  <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
 <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
 >
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
 >
 >
 >
 >
 __________________________________________________________________________
 > OpenStack Development Mailing List (not for usage questions)
 > Unsubscribe:
 OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
 <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
 > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
 >

 __________________________________________________________________________
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
 <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Nov 2, 2017 in openstack-dev by Dmitry_Tantsur (18,080 points)   2 3 4

11 Responses

0 votes

And the winner is Mon, 30 Oct, 2pm UTC!

The bluejeans ID is https://bluejeans.com/757528759
(works without plugins in recent FF and Chrome; if it asks to install an app,
ignore it and look for a link saying "join with browser")

On 10/23/2017 05:02 PM, Dmitry Tantsur wrote:
Hi all!

I'd like to invite you to the discussion of the way to implement traits in
ironic and the ironic virt driver. Please vote for the time at
https://doodle.com/poll/ts43k98kkvniv8uz. Please vote by EOD tomorrow.

Note that it's going to be a technical discussion - please make sure you
understand what traits are and why ironic cares about them. See below for more
context.

We'll probably use my bluejeans account, as it works without plugins in modern
browsers. I'll post a meeting ID when we pick the date.

On 10/23/2017 04:09 PM, Eric Fried wrote:

We discussed this a little bit further in IRC [1]. We're all in
agreement, but it's worth being precise on a couple of points:

  • We're distinguishing between a "feature" and the "trait" that
    represents it in placement. For the sake of this discussion, a
    "feature" can (maybe) be switched on or off, but a "trait" can either be
    present or absent on a RP.
  • It matters who can turn a feature on/off.

    • If it can be done by virt at spawn time, then it makes sense to have
      the trait on the RP, and you can switch the feature on/off via a
      separate extra_spec.
    • But if it's e.g. an admin action, and spawn has no control, then the
      trait needs to be added whenever the feature is on, and removed
      whenever the feature is off.

[1]
http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13

On 10/23/2017 08:15 AM, Sylvain Bauza wrote:

On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried <openstack@fried.cc
openstack@fried.cc> wrote:

  I agree with Sean.  In general terms:

  * A resource provider should be marked with a trait if that feature
    * Can be turned on or off (whether it's currently on or not); or
    * Is always on and can't ever be turned off.

No, traits are not boolean. If a resource provider stops providing a
capability, then the existing related trait should just be removed,
that's it.
If you see a trait, that's just means that the related capability for
the Resource Provider is supported, that's it too.

MHO.

-Sylvain

  * A consumer wanting that feature present (doesn't matter whether it's
  on or off) should specify it as a required *trait*.
  * A consumer wanting that feature present and turned on should
    * Specify it as a required trait; AND
    * Indicate that it be turned on via some other mechanism (e.g. a
  separate extra_spec).

  I believe this satisfies Dmitry's (Ironic's) needs, but also Jay's drive
  for placement purity.

  Please invite me to the hangout or whatever.

  Thanks,
  Eric

  On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
  >
  >
  >
  >
  > *From:*Jay Pipes [mailto:jaypipes@gmail.com
  <mailto:jaypipes@gmail.com>]
  > *Sent:* Monday, October 23, 2017 12:20 PM
  > *To:* OpenStack Development Mailing List
  <openstack-dev@lists.openstack.org
  <mailto:openstack-dev@lists.openstack.org>>
  > *Subject:* Re: [openstack-dev] [ironic] ironic and traits
  >
  >
  >
  > Writing from my phone... May I ask that before you proceed with any plan
  > that uses traits for state information that we have a hangout or
  > videoconference to discuss this? Unfortunately today and tomorrow I'm
  > not able to do a hangout but I can do one on Wednesday any time of the day.
  >
  >
  >
  > */[Mooney, Sean K] on the uefi boot topic I did bring up at the
  ptg that
  > we wanted to standardizes tratis for “verified boot” /*
  >
  > */that included a trait for uefi secure boot enabled and to
  indicated a
  > hardware root of trust, e.g. intel boot guard or similar/*
  >
  > */we distinctly wanted to be able to tag nova compute hosts with those
  > new traits so we could require that vms that request/*
  >
  > */a host with uefi secure boot enabled and a hardware root of
  trust are
  > scheduled only to those nodes. /*
  >
  > */ /*
  >
  > */There are many other examples that effect both vms and bare
  metal such
  > as, ecc/interleaved memory, cluster on die, /*
  >
  > */l3 cache code and data prioritization, vt-d/vt-c, HPET, Hyper
  > threading, power states … all of these feature may be present on the
  > platform/*
  >
  > */but I also need to know if they are turned on. Ruling out state in
  > traits means all of this logic will eventually get pushed to scheduler
  > filters/*
  >
  > */which will be suboptimal long term as more state is tracked.
  Software
  > defined infrastructure may be the future but hardware defined
  software/*
  >
  > */is sadly the present…/*
  >
  > */ /*
  >
  > */I do however think there should be a sperateion between asking for a
  > host that provides x with a trait and  asking for x to be
  configure via/*
  >
  > */A trait. The trait secure_boot_enabled should never result in the
  > feature being enabled It should just find a host with it on. If
  you want/*
  >
  > */To request it to be turned on you would request a host with
  > secure_boot_capable as a trait and have a flavor extra spec or image
  > property to request/*
  >
  > */Ironic to enabled it.  these are two very different request and
  should
  > not be treated the same. /*
  >
  >
  >
  >
  >
  > Lemme know!
  >
  > -jay
  >
  >
  >
  > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur" <dtantsur@redhat.com <mailto:dtantsur@redhat.com>
  > <mailto:dtantsur@redhat.com <mailto:dtantsur@redhat.com>>> wrote:
  >
  >     Hi Jay!
  >
  >     I appreciate your comments, but I think you're approaching the
  >     problem from purely VM point of view. Things simply don't work the
  >     same way in bare metal, at least not if we want to provide the same
  >     user experience.
  >
  >
  >
  >     On Sun, Oct 22, 2017 at 2:25 PM, Jay Pipes <jaypipes@gmail.com <mailto:jaypipes@gmail.com>
  >     <mailto:jaypipes@gmail.com <mailto:jaypipes@gmail.com>>> wrote:
  >
  >         Sorry for delay, took a week off before starting a new job.
  >         Comments inline.
  >
  >         On 10/16/2017 12:24 PM, Dmitry Tantsur wrote:
  >
  >             Hi all,
  >
  >             I promised John to dump my thoughts on traits to the
  ML, so
  >             here we go :)
  >
  >             I see two roles of traits (or kinds of traits) for
  bare metal:
  >             1. traits that say what the node can do already (e.g. "the
  >             node is
  >             doing UEFI boot")
  >             2. traits that say what the node can be *configured* to do
  >             (e.g. "the node can
  >             boot in UEFI mode")
  >
  >
  >         There's only one role for traits. #2 above. #1 is state
  >         information. Traits are not for state information. Traits are
  >         only for communicating capabilities of a resource provider
  >         (baremetal node).
  >
  >
  >
  >     These are not different, that's what I'm talking about here. No
  >     users care about the difference between "this node was put in UEFI
  >     mode by an operator in advance", "this node was put in UEFI
  mode by
  >     an ironic driver on demand" and "this node is always in UEFI mode,
  >     because it's AARCH64 and it does not have BIOS". These situation
  >     produce the same result (the node is booted in UEFI mode), and
  thus
  >     it's up to ironic to hide this difference.
  >
  >
  >
  >     My suggestion with traits is one way to do it, I'm not sure
  what you
  >     suggest though.
  >
  >
  >
  >
  >         For example, let's say we add the following to the os-traits
  >         library [1]
  >
  >         * STORAGE_RAID_0
  >         * STORAGE_RAID_1
  >         * STORAGE_RAID_5
  >         * STORAGE_RAID_6
  >         * STORAGE_RAID_10
  >
  >         The Ironic administrator would add all RAID-related traits to
  >         the baremetal nodes that had the *capability* of
  supporting that
  >         particular RAID setup [2]
  >
  >         When provisioned, the baremetal node would either have RAID
  >         configured in a certain level or not configured at all.
  >
  >
  >         A very important note: the Placement API and Nova
  scheduler (or
  >         future Ironic scheduler) doesn't care about this. At all.
  I know
  >         it sounds like I'm being callous, but I'm not. Placement and
  >         scheduling doesn't care about the state of things. It only
  cares
  >         about the capabilities of target destinations. That's it.
  >
  >
  >
  >     Yes, because VMs always start with a clean state, and
  hypervisor is
  >     there to ensure that. We don't have this luxury in ironic :) E.g.
  >     our SNMP driver is not even aware of boot modes (or RAID, or BIOS
  >     configuration), which does not mean that a node using it cannot be
  >     in UEFI mode (have a RAID or BIOS pre-configured, etc, etc).
  >
  >
  >
  >
  >
  >             This seems confusing, but it's actually very useful.
  Say, I
  >             have a flavor that
  >             requests UEFI boot via a trait. It will match both the
  nodes
  >             that are already in
  >             UEFI mode, as well as nodes that can be put in UEFI mode.
  >
  >
  >         No :) It will only match nodes that have the UEFI capability.
  >         The set of providers that have the ability to be booted
  via UEFI
  >         is *always* a superset of the set of providers that *have been
  >         booted via UEFI*. Placement and scheduling decisions only care
  >         about that superset -- the providers with a particular
  capability.
  >
  >
  >
  >     Well, no, it will. Again, you're purely basing on the VM idea,
  where
  >     a VM is always *put* in UEFI mode, no matter how the hypervisor
  >     looks like. It is simply not the case for us. You have to care
  what
  >     state the node is, because many drivers cannot change this state.
  >
  >
  >
  >
  >
  >             This idea goes further with deploy templates (new concept
  >             we've been thinking
  >             about). A flavor can request something like CUSTOM_RAID_5,
  >             and it will match the
  >             nodes that already have RAID 5, or, more
  interestingly, the
  >             nodes on which we
  >             can build RAID 5 before deployment. The UEFI example above
  >             can be treated in a
  >             similar way.
  >
  >             This ends up with two sources of knowledge about traits in
  >             ironic:
  >             1. Operators setting something they know about hardware
  >             ("this node is in UEFI
  >             mode"),
  >             2. Ironic drivers reporting something they
  >                2.1. know about hardware ("this node is in UEFI mode" -
  >             again)
  >                2.2. can do about hardware ("I can put this node in
  UEFI
  >             mode")
  >
  >
  >         You're correct that both pieces of information are important.
  >         However, only the "can do about hardware" part is relevant to
  >         Placement and Nova.
  >
  >             For case #1 we are planning on a new CRUD API to set/unset
  >             traits for a node.
  >
  >
  >         I would *strongly* advise against this. Traits are not for
  state
  >         information.
  >
  >         Instead, consider having a DB (or JSON) schema that lists
  state
  >         information in fields that are explicitly for that state
  >         information.
  >
  >         For example, a schema that looks like this:
  >
  >         {
  >           "boot": {
  >             "mode": <one of 'bios' or 'uefi'>,
  >             "params": <dict>
  >           },
  >           "disk": {
  >             "raid": {
  >               "level": <int>,
  >               "controller": <one of 'sw' or 'hw'>,
  >               "driver": <string>,
  >               "params": <dict>
  >             },  ...
  >           },
  >           "network": {
  >             ...
  >           }
  >         }
  >
  >         etc, etc.
  >
  >         Don't use trait strings to represent state information.
  >
  >
  >
  >     I don't see an alternative proposal that will satisfy what we have
  >     to solve.
  >
  >
  >
  >
  >         Best,
  >         -jay
  >
  >             Case #2 is more interesting. We have two options, I think:
  >
  >             a) Operators still set traits on nodes, drivers are simply
  >             validating them. E.g.
  >             an operators sets CUSTOM_RAID_5, and the node's RAID
  >             interface checks if it is
  >             possible to do. The downside is obvious - with a lot of
  >             deploy templates
  >             available it can be a lot of manual work.
  >
  >             b) Drivers report the traits, and they get somehow
  added to
  >             the traits provided
  >             by an operator. Technically, there are sub-cases again:
  >                b.1) The new traits API returns a union of
  >             operator-provided and
  >             driver-provided traits
  >                b.2) The new traits API returns only operator-provided
  >             traits; driver-provided
  >             traits are returned e.g. via a new field
  >             (node.driver_traits). Then nova will
  >             have to merge the lists itself.
  >
  >             My personal favorite is the last option: I'd like a clear
  >             distinction between
  >             different "sources" of traits, but I'd also like to reduce
  >             manual work for
  >             operators.
  >
  >             A valid counter-argument is: what if an operator wants to
  >             override a
  >             driver-provided trait? E.g. a node can do RAID 5, but I
  >             don't want this
  >             particular node to do it for any reason. I'm not sure if
  >             it's a valid case, and
  >             what to do about it.
  >
  >             Let me know what you think.
  >
  >             Dmitry
  >
  >
  >         [1]
  http://git.openstack.org/cgit/openstack/os-traits/tree/
  
  >         [2] Based on how many attached disks the node had, the
  presence
  >         and abilities of a hardware RAID controller, etc
  >
  >
  >
  >
   __________________________________________________________________________
  >         OpenStack Development Mailing List (not for usage questions)
  >         Unsubscribe:
  >
   OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
  
  >
   >>       >
  >         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  >
  >
  >
  >
  >     __________________________________________________________________________
  >     OpenStack Development Mailing List (not for usage questions)
  >     Unsubscribe:
  >     OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
  
  >
   >>       >
  >
   http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  >
  >
  >
  >
  __________________________________________________________________________
  > OpenStack Development Mailing List (not for usage questions)
  > Unsubscribe:
  OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
  
  > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  >

  __________________________________________________________________________
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
  OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
  
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 24, 2017 by Dmitry_Tantsur (18,080 points)   2 3 4
0 votes

Sigh, sorry. I forgot that we're moving back to winter time this weekend. I
think the time is 3pm UTC then. It seems to be 11am eastern US:
https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et.

On 10/24/2017 06:00 PM, Dmitry Tantsur wrote:
And the winner is Mon, 30 Oct, 2pm UTC!

The bluejeans ID is https://bluejeans.com/757528759
(works without plugins in recent FF and Chrome; if it asks to install an app,
ignore it and look for a link saying "join with browser")

On 10/23/2017 05:02 PM, Dmitry Tantsur wrote:

Hi all!

I'd like to invite you to the discussion of the way to implement traits in
ironic and the ironic virt driver. Please vote for the time at
https://doodle.com/poll/ts43k98kkvniv8uz. Please vote by EOD tomorrow.

Note that it's going to be a technical discussion - please make sure you
understand what traits are and why ironic cares about them. See below for more
context.

We'll probably use my bluejeans account, as it works without plugins in modern
browsers. I'll post a meeting ID when we pick the date.

On 10/23/2017 04:09 PM, Eric Fried wrote:

We discussed this a little bit further in IRC [1].  We're all in
agreement, but it's worth being precise on a couple of points:

  • We're distinguishing between a "feature" and the "trait" that
    represents it in placement.  For the sake of this discussion, a
    "feature" can (maybe) be switched on or off, but a "trait" can either be
    present or absent on a RP.
  • It matters who can turn a feature on/off.
        * If it can be done by virt at spawn time, then it makes sense to have
    the trait on the RP, and you can switch the feature on/off via a
    separate extra_spec.
        * But if it's e.g. an admin action, and spawn has no control, then the
    trait needs to be *added* whenever the feature is *on*, and *removed*
    whenever the feature is *off*.

[1]
http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13

On 10/23/2017 08:15 AM, Sylvain Bauza wrote:

On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried <openstack@fried.cc
openstack@fried.cc> wrote:

      I agree with Sean.  In general terms:

      * A resource provider should be marked with a trait if that feature
        * Can be turned on or off (whether it's currently on or not); or
        * Is always on and can't ever be turned off.

No, traits are not boolean. If a resource provider stops providing a
capability, then the existing related trait should just be removed,
that's it.
If you see a trait, that's just means that the related capability for
the Resource Provider is supported, that's it too.

MHO.

-Sylvain

      * A consumer wanting that feature present (doesn't matter whether it's
      on or off) should specify it as a required trait.
      * A consumer wanting that feature present and turned on should
        * Specify it as a required trait; AND
        * Indicate that it be turned on via some other mechanism (e.g. a
      separate extra_spec).

      I believe this satisfies Dmitry's (Ironic's) needs, but also Jay's drive
      for placement purity.

      Please invite me to the hangout or whatever.

      Thanks,
      Eric

      On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
      >
      >
      >
      >
      > From:Jay Pipes [mailto:jaypipes@gmail.com
      jaypipes@gmail.com]
      > Sent: Monday, October 23, 2017 12:20 PM
      > To: OpenStack Development Mailing List
      <openstack-dev@lists.openstack.org
      openstack-dev@lists.openstack.org>
      > Subject: Re: [openstack-dev] [ironic] ironic and traits
      >
      >
      >
      > Writing from my phone... May I ask that before you proceed with any
plan
      > that uses traits for state information that we have a hangout or
      > videoconference to discuss this? Unfortunately today and tomorrow I'm
      > not able to do a hangout but I can do one on Wednesday any time of
the day.
      >
      >
      >
      > /[Mooney, Sean K] on the uefi boot topic I did bring up at the
      ptg that
      > we wanted to standardizes tratis for “verified boot” /

      >
      > /that included a trait for uefi secure boot enabled and to
      indicated a
      > hardware root of trust, e.g. intel boot guard or similar/

      >
      > /we distinctly wanted to be able to tag nova compute hosts with those
      > new traits so we could require that vms that request/

      >
      > /a host with uefi secure boot enabled and a hardware root of
      trust are
      > scheduled only to those nodes. /

      >
      > / /
      >
      > /There are many other examples that effect both vms and bare
      metal such
      > as, ecc/interleaved memory, cluster on die, /

      >
      > /l3 cache code and data prioritization, vt-d/vt-c, HPET, Hyper
      > threading, power states … all of these feature may be present on the
      > platform/

      >
      > /but I also need to know if they are turned on. Ruling out state in
      > traits means all of this logic will eventually get pushed to scheduler
      > filters/

      >
      > /which will be suboptimal long term as more state is tracked.
      Software
      > defined infrastructure may be the future but hardware defined
      software/

      >
      > /is sadly the present…/
      >
      > / /
      >
      > /I do however think there should be a sperateion between asking for a
      > host that provides x with a trait and  asking for x to be
      configure via/

      >
      > /A trait. The trait securebootenabled should never result in the
      > feature being enabled It should just find a host with it on. If
      you want/

      >
      > /To request it to be turned on you would request a host with
      > securebootcapable as a trait and have a flavor extra spec or image
      > property to request/

      >
      > /Ironic to enabled it.  these are two very different request and
      should
      > not be treated the same. /

      >
      >
      >
      >
      >
      > Lemme know!
      >
      > -jay
      >
      >
      >
      > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur" <dtantsur@redhat.com
dtantsur@redhat.com
      > <mailto:dtantsur@redhat.com dtantsur@redhat.com>> wrote:
      >
      >     Hi Jay!
      >
      >     I appreciate your comments, but I think you're approaching the
      >     problem from purely VM point of view. Things simply don't work the
      >     same way in bare metal, at least not if we want to provide the same
      >     user experience.
      >
      >
      >
      >     On Sun, Oct 22, 2017 at 2:25 PM, Jay Pipes <jaypipes@gmail.com
jaypipes@gmail.com
      >     <mailto:jaypipes@gmail.com jaypipes@gmail.com>> wrote:
      >
      >         Sorry for delay, took a week off before starting a new job.
      >         Comments inline.
      >
      >         On 10/16/2017 12:24 PM, Dmitry Tantsur wrote:
      >
      >             Hi all,
      >
      >             I promised John to dump my thoughts on traits to the
      ML, so
      >             here we go :)
      >
      >             I see two roles of traits (or kinds of traits) for
      bare metal:
      >             1. traits that say what the node can do already (e.g. "the
      >             node is
      >             doing UEFI boot")
      >             2. traits that say what the node can be configured to do
      >             (e.g. "the node can
      >             boot in UEFI mode")
      >
      >
      >         There's only one role for traits. #2 above. #1 is state
      >         information. Traits are not for state information. Traits are
      >         only for communicating capabilities of a resource provider
      >         (baremetal node).
      >
      >
      >
      >     These are not different, that's what I'm talking about here. No
      >     users care about the difference between "this node was put in UEFI
      >     mode by an operator in advance", "this node was put in UEFI
      mode by
      >     an ironic driver on demand" and "this node is always in UEFI mode,
      >     because it's AARCH64 and it does not have BIOS". These situation
      >     produce the same result (the node is booted in UEFI mode), and
      thus
      >     it's up to ironic to hide this difference.
      >
      >
      >
      >     My suggestion with traits is one way to do it, I'm not sure
      what you
      >     suggest though.
      >
      >
      >
      >
      >         For example, let's say we add the following to the os-traits
      >         library [1]
      >
      >         * STORAGERAID0
      >         * STORAGERAID1
      >         * STORAGERAID5
      >         * STORAGERAID6
      >         * STORAGERAID10
      >
      >         The Ironic administrator would add all RAID-related traits to
      >         the baremetal nodes that had the capability of
      supporting that
      >         particular RAID setup [2]
      >
      >         When provisioned, the baremetal node would either have RAID
      >         configured in a certain level or not configured at all.
      >
      >
      >         A very important note: the Placement API and Nova
      scheduler (or
      >         future Ironic scheduler) doesn't care about this. At all.
      I know
      >         it sounds like I'm being callous, but I'm not. Placement and
      >         scheduling doesn't care about the state of things. It only
      cares
      >         about the capabilities of target destinations. That's it.
      >
      >
      >
      >     Yes, because VMs always start with a clean state, and
      hypervisor is
      >     there to ensure that. We don't have this luxury in ironic :) E.g.
      >     our SNMP driver is not even aware of boot modes (or RAID, or BIOS
      >     configuration), which does not mean that a node using it cannot be
      >     in UEFI mode (have a RAID or BIOS pre-configured, etc, etc).
      >
      >
      >
      >
      >
      >             This seems confusing, but it's actually very useful.
      Say, I
      >             have a flavor that
      >             requests UEFI boot via a trait. It will match both the
      nodes
      >             that are already in
      >             UEFI mode, as well as nodes that can be put in UEFI mode.
      >
      >
      >         No :) It will only match nodes that have the UEFI capability.
      >         The set of providers that have the ability to be booted
      via UEFI
      >         is always a superset of the set of providers that have been
      >         booted via UEFI
. Placement and scheduling decisions only care
      >         about that superset -- the providers with a particular
      capability.
      >
      >
      >
      >     Well, no, it will. Again, you're purely basing on the VM idea,
      where
      >     a VM is always put in UEFI mode, no matter how the hypervisor
      >     looks like. It is simply not the case for us. You have to care
      what
      >     state the node is, because many drivers cannot change this state.
      >
      >
      >
      >
      >
      >             This idea goes further with deploy templates (new concept
      >             we've been thinking
      >             about). A flavor can request something like CUSTOMRAID5,
      >             and it will match the
      >             nodes that already have RAID 5, or, more
      interestingly, the
      >             nodes on which we
      >             can build RAID 5 before deployment. The UEFI example above
      >             can be treated in a
      >             similar way.
      >
      >             This ends up with two sources of knowledge about traits in
      >             ironic:
      >             1. Operators setting something they know about hardware
      >             ("this node is in UEFI
      >             mode"),
      >             2. Ironic drivers reporting something they
      >                2.1. know about hardware ("this node is in UEFI mode" -
      >             again)
      >                2.2. can do about hardware ("I can put this node in
      UEFI
      >             mode")
      >
      >
      >         You're correct that both pieces of information are important.
      >         However, only the "can do about hardware" part is relevant to
      >         Placement and Nova.
      >
      >             For case #1 we are planning on a new CRUD API to set/unset
      >             traits for a node.
      >
      >
      >         I would strongly advise against this. Traits are not for
      state
      >         information.
      >
      >         Instead, consider having a DB (or JSON) schema that lists
      state
      >         information in fields that are explicitly for that state
      >         information.
      >
      >         For example, a schema that looks like this:
      >
      >         {
      >           "boot": {
      >             "mode": ,
      >             "params":
      >           },
      >           "disk": {
      >             "raid": {
      >               "level": ,
      >               "controller": ,
      >               "driver": ,
      >               "params":
      >             },  ...
      >           },
      >           "network": {
      >             ...
      >           }
      >         }
      >
      >         etc, etc.
      >
      >         Don't use trait strings to represent state information.
      >
      >
      >
      >     I don't see an alternative proposal that will satisfy what we have
      >     to solve.
      >
      >
      >
      >
      >         Best,
      >         -jay
      >
      >             Case #2 is more interesting. We have two options, I think:
      >
      >             a) Operators still set traits on nodes, drivers are simply
      >             validating them. E.g.
      >             an operators sets CUSTOMRAID5, and the node's RAID
      >             interface checks if it is
      >             possible to do. The downside is obvious - with a lot of
      >             deploy templates
      >             available it can be a lot of manual work.
      >
      >             b) Drivers report the traits, and they get somehow
      added to
      >             the traits provided
      >             by an operator. Technically, there are sub-cases again:
      >                b.1) The new traits API returns a union of
      >             operator-provided and
      >             driver-provided traits
      >                b.2) The new traits API returns only operator-provided
      >             traits; driver-provided
      >             traits are returned e.g. via a new field
      >             (node.driver_traits). Then nova will
      >             have to merge the lists itself.
      >
      >             My personal favorite is the last option: I'd like a clear
      >             distinction between
      >             different "sources" of traits, but I'd also like to reduce
      >             manual work for
      >             operators.
      >
      >             A valid counter-argument is: what if an operator wants to
      >             override a
      >             driver-provided trait? E.g. a node can do RAID 5, but I
      >             don't want this
      >             particular node to do it for any reason. I'm not sure if
      >             it's a valid case, and
      >             what to do about it.
      >
      >             Let me know what you think.
      >
      >             Dmitry
      >
      >
      >         [1]
      http://git.openstack.org/cgit/openstack/os-traits/tree/
     
      >         [2] Based on how many attached disks the node had, the
      presence
      >         and abilities of a hardware RAID controller, etc
      >
      >
      >
      >

      >         OpenStack Development Mailing List (not for usage questions)
      >         Unsubscribe:
      >
       OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
     
      >
       >>>       >
      >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
     
      >
      >
      >
      >
      >
      >     OpenStack Development Mailing List (not for usage questions)
      >     Unsubscribe:
      >     OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
     
      >
       >>>       >
      >
       http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
     
      >
      >
      >
      >

      > OpenStack Development Mailing List (not for usage questions)
      > Unsubscribe:
      OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
     
      > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
     
      >

  


      OpenStack Development Mailing List (not for usage questions)
      Unsubscribe:
      OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
     
      http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
     


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 24, 2017 by Dmitry_Tantsur (18,080 points)   2 3 4
0 votes

Aaaand sorry again, but due to sudden errands I won't be able to attend.
Please feel free to use my bluejeans room anyway. I think my position on
traits is more or less clear from previous discussions with John, Sam and
Eric.

2017-10-24 18:07 GMT+02:00 Dmitry Tantsur dtantsur@redhat.com:

Sigh, sorry. I forgot that we're moving back to winter time this weekend.
I think the time is 3pm UTC then. It seems to be 11am eastern US:
https://www.timeanddate.com/worldclock/converter.html?iso=20
171030T150000&p1=37&p2=tz_et.

On 10/24/2017 06:00 PM, Dmitry Tantsur wrote:

And the winner is Mon, 30 Oct, 2pm UTC!

The bluejeans ID is https://bluejeans.com/757528759
(works without plugins in recent FF and Chrome; if it asks to install an
app, ignore it and look for a link saying "join with browser")

On 10/23/2017 05:02 PM, Dmitry Tantsur wrote:

Hi all!

I'd like to invite you to the discussion of the way to implement traits
in
ironic and the ironic virt driver. Please vote for the time at
https://doodle.com/poll/ts43k98kkvniv8uz. Please vote by EOD tomorrow.

Note that it's going to be a technical discussion - please make sure you
understand what traits are and why ironic cares about them. See below
for more
context.

We'll probably use my bluejeans account, as it works without plugins in
modern
browsers. I'll post a meeting ID when we pick the date.

On 10/23/2017 04:09 PM, Eric Fried wrote:

We discussed this a little bit further in IRC [1]. We're all in
agreement, but it's worth being precise on a couple of points:

  • We're distinguishing between a "feature" and the "trait" that
    represents it in placement. For the sake of this discussion, a
    "feature" can (maybe) be switched on or off, but a "trait" can either be
    present or absent on a RP.
  • It matters who can turn a feature on/off.

    • If it can be done by virt at spawn time, then it makes sense to
      have
      the trait on the RP, and you can switch the feature on/off via a
      separate extra_spec.
    • But if it's e.g. an admin action, and spawn has no control, then
      the
      trait needs to be added whenever the feature is on, and removed
      whenever the feature is off.

[1]
http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%
23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13

On 10/23/2017 08:15 AM, Sylvain Bauza wrote:

On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried <openstack@fried.cc
openstack@fried.cc> wrote:

  I agree with Sean.  In general terms:

  * A resource provider should be marked with a trait if that

feature
* Can be turned on or off (whether it's currently on or not);
or
* Is always on and can't ever be turned off.

No, traits are not boolean. If a resource provider stops providing a
capability, then the existing related trait should just be removed,
that's it.
If you see a trait, that's just means that the related capability for
the Resource Provider is supported, that's it too.

MHO.

-Sylvain

  * A consumer wanting that feature present (doesn't matter

whether it's
on or off) should specify it as a required trait.
* A consumer wanting that feature present and turned on should
* Specify it as a required trait; AND
* Indicate that it be turned on via some other mechanism (e.g.
a
separate extra_spec).

  I believe this satisfies Dmitry's (Ironic's) needs, but also

Jay's drive
for placement purity.

  Please invite me to the hangout or whatever.

  Thanks,
  Eric

  On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
  >
  >
  >
  >
  > *From:*Jay Pipes [mailto:jaypipes@gmail.com
  <mailto:jaypipes@gmail.com>]
  > *Sent:* Monday, October 23, 2017 12:20 PM
  > *To:* OpenStack Development Mailing List
  <openstack-dev@lists.openstack.org
  <mailto:openstack-dev@lists.openstack.org>>
  > *Subject:* Re: [openstack-dev] [ironic] ironic and traits
  >
  >
  >
  > Writing from my phone... May I ask that before you proceed

with any plan

that uses traits for state information that we have a hangout
or
videoconference to discuss this? Unfortunately today and
tomorrow I'm
not able to do a hangout but I can do one on Wednesday any
time of the day.

/[Mooney, Sean K] on the uefi boot topic I did bring up at the
ptg that
we wanted to standardizes tratis for “verified boot” /

/that included a trait for uefi secure boot enabled and to
indicated a
hardware root of trust, e.g. intel boot guard or similar/

/we distinctly wanted to be able to tag nova compute hosts
with those
new traits so we could require that vms that request/

/a host with uefi secure boot enabled and a hardware root of
trust are
scheduled only to those nodes. /

/ /

/There are many other examples that effect both vms and bare
metal such
as, ecc/interleaved memory, cluster on die, /

/l3 cache code and data prioritization, vt-d/vt-c, HPET, Hyper
threading, power states … all of these feature may be present
on the
platform/

/but I also need to know if they are turned on. Ruling out
state in
traits means all of this logic will eventually get pushed to
scheduler
filters/

/which will be suboptimal long term as more state is tracked.
Software
defined infrastructure may be the future but hardware defined
software/

/is sadly the present…/

/ /

/I do however think there should be a sperateion between
asking for a
host that provides x with a trait and asking for x to be
configure via/

/A trait. The trait securebootenabled should never result
in the
feature being enabled It should just find a host with it on. If
you want/

/To request it to be turned on you would request a host with
securebootcapable as a trait and have a flavor extra spec or
image
property to request/

/Ironic to enabled it. these are two very different request
and
should
not be treated the same. /

Lemme know!

-jay

On Oct 23, 2017 5:01 AM, "Dmitry Tantsur" <dtantsur@redhat.com
dtantsur@redhat.com
<mailto:dtantsur@redhat.com dtantsur@redhat.com>>
wrote:

Hi Jay!

I appreciate your comments, but I think you're approaching

the
problem from purely VM point of view. Things simply don't
work the
same way in bare metal, at least not if we want to provide
the same
user experience.

On Sun, Oct 22, 2017 at 2:25 PM, Jay Pipes <

jaypipes@gmail.com jaypipes@gmail.com
<mailto:jaypipes@gmail.com jaypipes@gmail.com>>
wrote:

    Sorry for delay, took a week off before starting a new

job.
Comments inline.

    On 10/16/2017 12:24 PM, Dmitry Tantsur wrote:

        Hi all,

        I promised John to dump my thoughts on traits to

the
ML, so
here we go :)

        I see two roles of traits (or kinds of traits) for
  bare metal:
        1. traits that say what the node can do already

(e.g. "the
node is
doing UEFI boot")
2. traits that say what the node can be
configured to do
(e.g. "the node can
boot in UEFI mode")

    There's only one role for traits. #2 above. #1 is state
    information. Traits are not for state information.

Traits are
only for communicating capabilities of a resource
provider
(baremetal node).

These are not different, that's what I'm talking about

here. No
users care about the difference between "this node was put
in UEFI
mode by an operator in advance", "this node was put in UEFI
mode by
an ironic driver on demand" and "this node is always in
UEFI mode,
because it's AARCH64 and it does not have BIOS". These
situation
produce the same result (the node is booted in UEFI mode),
and
thus
it's up to ironic to hide this difference.

My suggestion with traits is one way to do it, I'm not sure
  what you
suggest though.




    For example, let's say we add the following to the

os-traits
library [1]

    * STORAGE_RAID_0
    * STORAGE_RAID_1
    * STORAGE_RAID_5
    * STORAGE_RAID_6
    * STORAGE_RAID_10

    The Ironic administrator would add all RAID-related

traits to
the baremetal nodes that had the capability of
supporting that
particular RAID setup [2]

    When provisioned, the baremetal node would either have

RAID
configured in a certain level or not configured at all.

    A very important note: the Placement API and Nova
  scheduler (or
    future Ironic scheduler) doesn't care about this. At

all.
I know
it sounds like I'm being callous, but I'm not.
Placement and
scheduling doesn't care about the state of things. It
only
cares
about the capabilities of target destinations. That's
it.

Yes, because VMs always start with a clean state, and
  hypervisor is
there to ensure that. We don't have this luxury in ironic

:) E.g.
our SNMP driver is not even aware of boot modes (or RAID,
or BIOS
configuration), which does not mean that a node using it
cannot be
in UEFI mode (have a RAID or BIOS pre-configured, etc,
etc).

        This seems confusing, but it's actually very

useful.
Say, I
have a flavor that
requests UEFI boot via a trait. It will match both
the
nodes
that are already in
UEFI mode, as well as nodes that can be put in
UEFI mode.

    No :) It will only match nodes that have the UEFI

capability.
The set of providers that have the ability to be booted
via UEFI
is always a superset of the set of providers that
have been
booted via UEFI
. Placement and scheduling decisions
only care
about that superset -- the providers with a particular
capability.

Well, no, it will. Again, you're purely basing on the VM

idea,
where
a VM is always put in UEFI mode, no matter how the
hypervisor
looks like. It is simply not the case for us. You have to
care
what
state the node is, because many drivers cannot change this
state.

        This idea goes further with deploy templates (new

concept
we've been thinking
about). A flavor can request something like
CUSTOMRAID5,
and it will match the
nodes that already have RAID 5, or, more
interestingly, the
nodes on which we
can build RAID 5 before deployment. The UEFI
example above
can be treated in a
similar way.

        This ends up with two sources of knowledge about

traits in
ironic:
1. Operators setting something they know about
hardware
("this node is in UEFI
mode"),
2. Ironic drivers reporting something they
2.1. know about hardware ("this node is in UEFI
mode" -
again)
2.2. can do about hardware ("I can put this
node in
UEFI
mode")

    You're correct that both pieces of information are

important.
However, only the "can do about hardware" part is
relevant to
Placement and Nova.

        For case #1 we are planning on a new CRUD API to

set/unset
traits for a node.

    I would *strongly* advise against this. Traits are not

for
state
information.

    Instead, consider having a DB (or JSON) schema that

lists
state
information in fields that are explicitly for that
state
information.

    For example, a schema that looks like this:

    {
      "boot": {
        "mode": <one of 'bios' or 'uefi'>,
        "params": <dict>
      },
      "disk": {
        "raid": {
          "level": <int>,
          "controller": <one of 'sw' or 'hw'>,
          "driver": <string>,
          "params": <dict>
        },  ...
      },
      "network": {
        ...
      }
    }

    etc, etc.

    Don't use trait strings to represent state information.



I don't see an alternative proposal that will satisfy what

we have
to solve.

    Best,
    -jay

        Case #2 is more interesting. We have two options,

I think:

        a) Operators still set traits on nodes, drivers

are simply
validating them. E.g.
an operators sets CUSTOMRAID5, and the node's
RAID
interface checks if it is
possible to do. The downside is obvious - with a
lot of
deploy templates
available it can be a lot of manual work.

        b) Drivers report the traits, and they get somehow
  added to
        the traits provided
        by an operator. Technically, there are sub-cases

again:
b.1) The new traits API returns a union of
operator-provided and
driver-provided traits
b.2) The new traits API returns only
operator-provided
traits; driver-provided
traits are returned e.g. via a new field
(node.driver_traits). Then nova will
have to merge the lists itself.

        My personal favorite is the last option: I'd like

a clear
distinction between
different "sources" of traits, but I'd also like
to reduce
manual work for
operators.

        A valid counter-argument is: what if an operator

wants to
override a
driver-provided trait? E.g. a node can do RAID 5,
but I
don't want this
particular node to do it for any reason. I'm not
sure if
it's a valid case, and
what to do about it.

        Let me know what you think.

        Dmitry


    [1]
  http://git.openstack.org/cgit/openstack/os-traits/tree/

    [2] Based on how many attached disks the node had, the
  presence
    and abilities of a hardware RAID controller, etc

___________________________________________________________


    OpenStack Development Mailing List (not for usage

questions)
Unsubscribe:

OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

subscribe>

subscribe
subscribe>>
http://lists.openstack.org/cg
i-bin/mailman/listinfo/openstack-dev
ck-dev>

_____________________________


OpenStack Development Mailing List (not for usage
questions)
Unsubscribe:
OpenStack-dev-request@lists.op
enstack.org?subject:unsubscribe

subscribe>

subscribe
subscribe>>

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac
k-dev

ck-dev>

____________________________________________________________


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

  ____________________________________________________________


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe



OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.op
enstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.op
enstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
--
-- Dmitry Tantsur
--


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 30, 2017 by divius.inside_at_gma (1,080 points)   1
0 votes

I'd prefer to have you on the call, Dima. How about we push it back to
tomorrow at the same time?

Can everyone make it then?

-jay

On 10/30/2017 10:11 AM, Dmitry Tantsur wrote:
Aaaand sorry again, but due to sudden errands I won't be able to attend.
Please feel free to use my bluejeans room anyway. I think my position on
traits is more or less clear from previous discussions with John, Sam
and Eric.

2017-10-24 18:07 GMT+02:00 Dmitry Tantsur <dtantsur@redhat.com
dtantsur@redhat.com>:

Sigh, sorry. I forgot that we're moving back to winter time this
weekend. I *think* the time is 3pm UTC then. It seems to be 11am
eastern US:
https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et
<https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et>.


On 10/24/2017 06:00 PM, Dmitry Tantsur wrote:

    And the winner is Mon, 30 Oct, 2pm UTC!

    The bluejeans ID is https://bluejeans.com/757528759
    <https://bluejeans.com/757528759>
    (works without plugins in recent FF and Chrome; if it asks to
    install an app, ignore it and look for a link saying "join with
    browser")

    On 10/23/2017 05:02 PM, Dmitry Tantsur wrote:

        Hi all!

        I'd like to invite you to the discussion of the way to
        implement traits in
        ironic and the ironic virt driver. Please vote for the time at
        https://doodle.com/poll/ts43k98kkvniv8uz
        <https://doodle.com/poll/ts43k98kkvniv8uz>. Please vote by
        EOD tomorrow.

        Note that it's going to be a technical discussion - please
        make sure you
        understand what traits are and why ironic cares about them.
        See below for more
        context.

        We'll probably use my bluejeans account, as it works without
        plugins in modern
        browsers. I'll post a meeting ID when we pick the date.


        On 10/23/2017 04:09 PM, Eric Fried wrote:

            We discussed this a little bit further in IRC [1]. 
            We're all in
            agreement, but it's worth being precise on a couple of
            points:

            * We're distinguishing between a "feature" and the
            "trait" that
            represents it in placement.  For the sake of this
            discussion, a
            "feature" can (maybe) be switched on or off, but a
            "trait" can either be
            present or absent on a RP.
            * It matters *who* can turn a feature on/off.
                 * If it can be done by virt at spawn time, then it
            makes sense to have
            the trait on the RP, and you can switch the feature
            on/off via a
            separate extra_spec.
                 * But if it's e.g. an admin action, and spawn has
            no control, then the
            trait needs to be *added* whenever the feature is *on*,
            and *removed*
            whenever the feature is *off*.

            [1]
            http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13
            


            On 10/23/2017 08:15 AM, Sylvain Bauza wrote:



                On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried
                <openstack@fried.cc
                <mailto:openstack@fried.cc
                <mailto:openstack@fried.cc>>> wrote:

                       I agree with Sean.  In general terms:

                       * A resource provider should be marked with a
                trait if that feature
                         * Can be turned on or off (whether it's
                currently on or not); or
                         * Is always on and can't ever be turned off.


                No, traits are not boolean. If a resource provider
                stops providing a
                capability, then the existing related trait should
                just be removed,
                that's it.
                If you see a trait, that's just means that the
                related capability for
                the Resource Provider is supported, that's it too.

                MHO.

                -Sylvain



                       * A consumer wanting that feature present
                (doesn't matter whether it's
                       on or off) should specify it as a required
                *trait*.
                       * A consumer wanting that feature present and
                turned on should
                         * Specify it as a required trait; AND
                         * Indicate that it be turned on via some
                other mechanism (e.g. a
                       separate extra_spec).

                       I believe this satisfies Dmitry's (Ironic's)
                needs, but also Jay's drive
                       for placement purity.

                       Please invite me to the hangout or whatever.

                       Thanks,
                       Eric

                       On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
                       >
                       >
                       >
                       >
                       > *From:*Jay Pipes [mailto:jaypipes@gmail.com
                <mailto:jaypipes@gmail.com>
                       <mailto:jaypipes@gmail.com
                <mailto:jaypipes@gmail.com>>]
                       > *Sent:* Monday, October 23, 2017 12:20 PM
                       > *To:* OpenStack Development Mailing List
                       <openstack-dev@lists.openstack.org
                <mailto:openstack-dev@lists.openstack.org>
                       <mailto:openstack-dev@lists.openstack.org
                <mailto:openstack-dev@lists.openstack.org>>>
                       > *Subject:* Re: [openstack-dev] [ironic]
                ironic and traits
                       >
                       >
                       >
                       > Writing from my phone... May I ask that
                before you proceed with any plan
                       > that uses traits for state information that
                we have a hangout or
                       > videoconference to discuss this?
                Unfortunately today and tomorrow I'm
                       > not able to do a hangout but I can do one
                on Wednesday any time of the day.
                       >
                       >
                       >
                       > */[Mooney, Sean K] on the uefi boot topic I
                did bring up at the
                       ptg that
                       > we wanted to standardizes tratis for
                “verified boot” /*
                       >
                       > */that included a trait for uefi secure
                boot enabled and to
                       indicated a
                       > hardware root of trust, e.g. intel boot
                guard or similar/*
                       >
                       > */we distinctly wanted to be able to tag
                nova compute hosts with those
                       > new traits so we could require that vms
                that request/*
                       >
                       > */a host with uefi secure boot enabled and
                a hardware root of
                       trust are
                       > scheduled only to those nodes. /*
                       >
                       > */ /*
                       >
                       > */There are many other examples that effect
                both vms and bare
                       metal such
                       > as, ecc/interleaved memory, cluster on die, /*
                       >
                       > */l3 cache code and data prioritization,
                vt-d/vt-c, HPET, Hyper
                       > threading, power states … all of these
                feature may be present on the
                       > platform/*
                       >
                       > */but I also need to know if they are
                turned on. Ruling out state in
                       > traits means all of this logic will
                eventually get pushed to scheduler
                       > filters/*
                       >
                       > */which will be suboptimal long term as
                more state is tracked.
                       Software
                       > defined infrastructure may be the future
                but hardware defined
                       software/*
                       >
                       > */is sadly the present…/*
                       >
                       > */ /*
                       >
                       > */I do however think there should be a
                sperateion between asking for a
                       > host that provides x with a trait and 
                asking for x to be
                       configure via/*
                       >
                       > */A trait. The trait secure_boot_enabled
                should never result in the
                       > feature being enabled It should just find a
                host with it on. If
                       you want/*
                       >
                       > */To request it to be turned on you would
                request a host with
                       > secure_boot_capable as a trait and have a
                flavor extra spec or image
                       > property to request/*
                       >
                       > */Ironic to enabled it.  these are two very
                different request and
                       should
                       > not be treated the same. /*
                       >
                       >
                       >
                       >
                       >
                       > Lemme know!
                       >
                       > -jay
                       >
                       >
                       >
                       > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur"
                <dtantsur@redhat.com <mailto:dtantsur@redhat.com>
                <mailto:dtantsur@redhat.com
                <mailto:dtantsur@redhat.com>>
                       > <mailto:dtantsur@redhat.com
                <mailto:dtantsur@redhat.com>
                <mailto:dtantsur@redhat.com
                <mailto:dtantsur@redhat.com>>>> wrote:
                       >
                       >     Hi Jay!
                       >
                       >     I appreciate your comments, but I think
                you're approaching the
                       >     problem from purely VM point of view.
                Things simply don't work the
                       >     same way in bare metal, at least not if
                we want to provide the same
                       >     user experience.
                       >
                       >
                       >
                       >     On Sun, Oct 22, 2017 at 2:25 PM, Jay
                Pipes <jaypipes@gmail.com
                <mailto:jaypipes@gmail.com>
                <mailto:jaypipes@gmail.com <mailto:jaypipes@gmail.com>>
                       >     <mailto:jaypipes@gmail.com
                <mailto:jaypipes@gmail.com>
                <mailto:jaypipes@gmail.com
                <mailto:jaypipes@gmail.com>>>> wrote:
                       >
                       >         Sorry for delay, took a week off
                before starting a new job.
                       >         Comments inline.
                       >
                       >         On 10/16/2017 12:24 PM, Dmitry
                Tantsur wrote:
                       >
                       >             Hi all,
                       >
                       >             I promised John to dump my
                thoughts on traits to the
                       ML, so
                       >             here we go :)
                       >
                       >             I see two roles of traits (or
                kinds of traits) for
                       bare metal:
                       >             1. traits that say what the
                node can do already (e.g. "the
                       >             node is
                       >             doing UEFI boot")
                       >             2. traits that say what the
                node can be *configured* to do
                       >             (e.g. "the node can
                       >             boot in UEFI mode")
                       >
                       >
                       >         There's only one role for traits.
                #2 above. #1 is state
                       >         information. Traits are not for
                state information. Traits are
                       >         only for communicating capabilities
                of a resource provider
                       >         (baremetal node).
                       >
                       >
                       >
                       >     These are not different, that's what
                I'm talking about here. No
                       >     users care about the difference between
                "this node was put in UEFI
                       >     mode by an operator in advance", "this
                node was put in UEFI
                       mode by
                       >     an ironic driver on demand" and "this
                node is always in UEFI mode,
                       >     because it's AARCH64 and it does not
                have BIOS". These situation
                       >     produce the same result (the node is
                booted in UEFI mode), and
                       thus
                       >     it's up to ironic to hide this difference.
                       >
                       >
                       >
                       >     My suggestion with traits is one way to
                do it, I'm not sure
                       what you
                       >     suggest though.
                       >
                       >
                       >
                       >
                       >         For example, let's say we add the
                following to the os-traits
                       >         library [1]
                       >
                       >         * STORAGE_RAID_0
                       >         * STORAGE_RAID_1
                       >         * STORAGE_RAID_5
                       >         * STORAGE_RAID_6
                       >         * STORAGE_RAID_10
                       >
                       >         The Ironic administrator would add
                all RAID-related traits to
                       >         the baremetal nodes that had the
                *capability* of
                       supporting that
                       >         particular RAID setup [2]
                       >
                       >         When provisioned, the baremetal
                node would either have RAID
                       >         configured in a certain level or
                not configured at all.
                       >
                       >
                       >         A very important note: the
                Placement API and Nova
                       scheduler (or
                       >         future Ironic scheduler) doesn't
                care about this. At all.
                       I know
                       >         it sounds like I'm being callous,
                but I'm not. Placement and
                       >         scheduling doesn't care about the
                state of things. It only
                       cares
                       >         about the capabilities of target
                destinations. That's it.
                       >
                       >
                       >
                       >     Yes, because VMs always start with a
                clean state, and
                       hypervisor is
                       >     there to ensure that. We don't have
                this luxury in ironic :) E.g.
                       >     our SNMP driver is not even aware of
                boot modes (or RAID, or BIOS
                       >     configuration), which does not mean
                that a node using it cannot be
                       >     in UEFI mode (have a RAID or BIOS
                pre-configured, etc, etc).
                       >
                       >
                       >
                       >
                       >
                       >             This seems confusing, but it's
                actually very useful.
                       Say, I
                       >             have a flavor that
                       >             requests UEFI boot via a trait.
                It will match both the
                       nodes
                       >             that are already in
                       >             UEFI mode, as well as nodes
                that can be put in UEFI mode.
                       >
                       >
                       >         No :) It will only match nodes that
                have the UEFI capability.
                       >         The set of providers that have the
                ability to be booted
                       via UEFI
                       >         is *always* a superset of the set
                of providers that *have been
                       >         booted via UEFI*. Placement and
                scheduling decisions only care
                       >         about that superset -- the
                providers with a particular
                       capability.
                       >
                       >
                       >
                       >     Well, no, it will. Again, you're purely
                basing on the VM idea,
                       where
                       >     a VM is always *put* in UEFI mode, no
                matter how the hypervisor
                       >     looks like. It is simply not the case
                for us. You have to care
                       what
                       >     state the node is, because many drivers
                cannot change this state.
                       >
                       >
                       >
                       >
                       >
                       >             This idea goes further with
                deploy templates (new concept
                       >             we've been thinking
                       >             about). A flavor can request
                something like CUSTOM_RAID_5,
                       >             and it will match the
                       >             nodes that already have RAID 5,
                or, more
                       interestingly, the
                       >             nodes on which we
                       >             can build RAID 5 before
                deployment. The UEFI example above
                       >             can be treated in a
                       >             similar way.
                       >
                       >             This ends up with two sources
                of knowledge about traits in
                       >             ironic:
                       >             1. Operators setting something
                they know about hardware
                       >             ("this node is in UEFI
                       >             mode"),
                       >             2. Ironic drivers reporting
                something they
                       >                2.1. know about hardware
                ("this node is in UEFI mode" -
                       >             again)
                       >                2.2. can do about hardware
                ("I can put this node in
                       UEFI
                       >             mode")
                       >
                       >
                       >         You're correct that both pieces of
                information are important.
                       >         However, only the "can do about
                hardware" part is relevant to
                       >         Placement and Nova.
                       >
                       >             For case #1 we are planning on
                a new CRUD API to set/unset
                       >             traits for a node.
                       >
                       >
                       >         I would *strongly* advise against
                this. Traits are not for
                       state
                       >         information.
                       >
                       >         Instead, consider having a DB (or
                JSON) schema that lists
                       state
                       >         information in fields that are
                explicitly for that state
                       >         information.
                       >
                       >         For example, a schema that looks
                like this:
                       >
                       >         {
                       >           "boot": {
                       >             "mode": <one of 'bios' or 'uefi'>,
                       >             "params": <dict>
                       >           },
                       >           "disk": {
                       >             "raid": {
                       >               "level": <int>,
                       >               "controller": <one of 'sw' or
                'hw'>,
                       >               "driver": <string>,
                       >               "params": <dict>
                       >             },  ...
                       >           },
                       >           "network": {
                       >             ...
                       >           }
                       >         }
                       >
                       >         etc, etc.
                       >
                       >         Don't use trait strings to
                represent state information.
                       >
                       >
                       >
                       >     I don't see an alternative proposal
                that will satisfy what we have
                       >     to solve.
                       >
                       >
                       >
                       >
                       >         Best,
                       >         -jay
                       >
                       >             Case #2 is more interesting. We
                have two options, I think:
                       >
                       >             a) Operators still set traits
                on nodes, drivers are simply
                       >             validating them. E.g.
                       >             an operators sets
                CUSTOM_RAID_5, and the node's RAID
                       >             interface checks if it is
                       >             possible to do. The downside is
                obvious - with a lot of
                       >             deploy templates
                       >             available it can be a lot of
                manual work.
                       >
                       >             b) Drivers report the traits,
                and they get somehow
                       added to
                       >             the traits provided
                       >             by an operator. Technically,
                there are sub-cases again:
                       >                b.1) The new traits API
                returns a union of
                       >             operator-provided and
                       >             driver-provided traits
                       >                b.2) The new traits API
                returns only operator-provided
                       >             traits; driver-provided
                       >             traits are returned e.g. via a
                new field
                       >             (node.driver_traits). Then nova
                will
                       >             have to merge the lists itself.
                       >
                       >             My personal favorite is the
                last option: I'd like a clear
                       >             distinction between
                       >             different "sources" of traits,
                but I'd also like to reduce
                       >             manual work for
                       >             operators.
                       >
                       >             A valid counter-argument is:
                what if an operator wants to
                       >             override a
                       >             driver-provided trait? E.g. a
                node can do RAID 5, but I
                       >             don't want this
                       >             particular node to do it for
                any reason. I'm not sure if
                       >             it's a valid case, and
                       >             what to do about it.
                       >
                       >             Let me know what you think.
                       >
                       >             Dmitry
                       >
                       >
                       >         [1]
                http://git.openstack.org/cgit/openstack/os-traits/tree/
                
                      
                                     >
                       >         [2] Based on how many attached
                disks the node had, the
                       presence
                       >         and abilities of a hardware RAID
                controller, etc
                       >
                       >
                       >
                       >
                      
                  __________________________________________________________________________
                       >         OpenStack Development Mailing List
                (not for usage questions)
                       >         Unsubscribe:
                       >
                OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                      
                                     >
                       >
                       
                                     
                      
                                     >>
                       >
                http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
                
                      
                                     >
                       >
                       >
                       >
                       >
                       >   
                  __________________________________________________________________________
                       >     OpenStack Development Mailing List (not
                for usage questions)
                       >     Unsubscribe:
                       >
                OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                      
                                     >
                       >
                       
                                     
                      
                                     >>
                       >
                http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
                
                      
                                     >
                       >
                       >
                       >
                       >
                      
                __________________________________________________________________________
                       > OpenStack Development Mailing List (not for
                usage questions)
                       > Unsubscribe:
                OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                      
                                     >
                       >
                http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
                
                      
                                     >
                       >

                      
                __________________________________________________________________________
                       OpenStack Development Mailing List (not for
                usage questions)
                       Unsubscribe:
                OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                      
                                     >
                http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
                
                      
                                     >




                __________________________________________________________________________
                OpenStack Development Mailing List (not for usage
                questions)
                Unsubscribe:
                OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
                


            __________________________________________________________________________
            OpenStack Development Mailing List (not for usage questions)
            Unsubscribe:
            OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
            
            http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
            




__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
--
-- Dmitry Tantsur
--


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 30, 2017 by Jay_Pipes (59,760 points)   3 7 13
0 votes

It's a holiday here tomorrow, but I don't have any specific plans, so I think
I'll be able to make it.

On 10/30/2017 03:13 PM, Jay Pipes wrote:
I'd prefer to have you on the call, Dima. How about we push it back to tomorrow
at the same time?

Can everyone make it then?

-jay

On 10/30/2017 10:11 AM, Dmitry Tantsur wrote:

Aaaand sorry again, but due to sudden errands I won't be able to attend.
Please feel free to use my bluejeans room anyway. I think my position on
traits is more or less clear from previous discussions with John, Sam and Eric.

2017-10-24 18:07 GMT+02:00 Dmitry Tantsur <dtantsur@redhat.com
dtantsur@redhat.com>:

    Sigh, sorry. I forgot that we're moving back to winter time this
    weekend. I think the time is 3pm UTC then. It seems to be 11am
    eastern US:

https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et


https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et.

    On 10/24/2017 06:00 PM, Dmitry Tantsur wrote:

        And the winner is Mon, 30 Oct, 2pm UTC!

        The bluejeans ID is https://bluejeans.com/757528759
        https://bluejeans.com/757528759
        (works without plugins in recent FF and Chrome; if it asks to
        install an app, ignore it and look for a link saying "join with
        browser")

        On 10/23/2017 05:02 PM, Dmitry Tantsur wrote:

            Hi all!

            I'd like to invite you to the discussion of the way to
            implement traits in
            ironic and the ironic virt driver. Please vote for the time at
            https://doodle.com/poll/ts43k98kkvniv8uz
            https://doodle.com/poll/ts43k98kkvniv8uz. Please vote by
            EOD tomorrow.

            Note that it's going to be a technical discussion - please
            make sure you
            understand what traits are and why ironic cares about them.
            See below for more
            context.

            We'll probably use my bluejeans account, as it works without
            plugins in modern
            browsers. I'll post a meeting ID when we pick the date.

            On 10/23/2017 04:09 PM, Eric Fried wrote:

                We discussed this a little bit further in IRC [1].
                We're all in
                agreement, but it's worth being precise on a couple of
                points:

                * We're distinguishing between a "feature" and the
                "trait" that
                represents it in placement.  For the sake of this
                discussion, a
                "feature" can (maybe) be switched on or off, but a
                "trait" can either be
                present or absent on a RP.
                * It matters who can turn a feature on/off.
                     * If it can be done by virt at spawn time, then it
                makes sense to have
                the trait on the RP, and you can switch the feature
                on/off via a
                separate extra_spec.
                     * But if it's e.g. an admin action, and spawn has
                no control, then the
                trait needs to be *added* whenever the feature is *on*,
                and *removed*
                whenever the feature is *off*.

                [1]

http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13


                On 10/23/2017 08:15 AM, Sylvain Bauza wrote:

                    On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried
                    <openstack@fried.cc
                    <mailto:openstack@fried.cc
                    openstack@fried.cc>> wrote:

                           I agree with Sean.  In general terms:

                           * A resource provider should be marked with a
                    trait if that feature
                             * Can be turned on or off (whether it's
                    currently on or not); or
                             * Is always on and can't ever be turned off.

                    No, traits are not boolean. If a resource provider
                    stops providing a
                    capability, then the existing related trait should
                    just be removed,
                    that's it.
                    If you see a trait, that's just means that the
                    related capability for
                    the Resource Provider is supported, that's it too.

                    MHO.

                    -Sylvain

                           * A consumer wanting that feature present
                    (doesn't matter whether it's
                           on or off) should specify it as a required
                    trait.
                           * A consumer wanting that feature present and
                    turned on should
                             * Specify it as a required trait; AND
                             * Indicate that it be turned on via some
                    other mechanism (e.g. a
                           separate extra_spec).

                           I believe this satisfies Dmitry's (Ironic's)
                    needs, but also Jay's drive
                           for placement purity.

                           Please invite me to the hangout or whatever.

                           Thanks,
                           Eric

                           On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
                           >
                           >
                           >
                           >
                           > From:Jay Pipes [mailto:jaypipes@gmail.com
                    jaypipes@gmail.com
                           <mailto:jaypipes@gmail.com
                    jaypipes@gmail.com>]
                           > Sent: Monday, October 23, 2017 12:20 PM
                           > To: OpenStack Development Mailing List
                           <openstack-dev@lists.openstack.org
                    openstack-dev@lists.openstack.org
                           <mailto:openstack-dev@lists.openstack.org
                    openstack-dev@lists.openstack.org>>
                           > Subject: Re: [openstack-dev] [ironic]
                    ironic and traits
                           >
                           >
                           >
                           > Writing from my phone... May I ask that
                    before you proceed with any plan
                           > that uses traits for state information that
                    we have a hangout or
                           > videoconference to discuss this?
                    Unfortunately today and tomorrow I'm
                           > not able to do a hangout but I can do one
                    on Wednesday any time of the day.
                           >
                           >
                           >
                           > /[Mooney, Sean K] on the uefi boot topic I
                    did bring up at the
                           ptg that
                           > we wanted to standardizes tratis for
                    “verified boot” /

                           >
                           > /that included a trait for uefi secure
                    boot enabled and to
                           indicated a
                           > hardware root of trust, e.g. intel boot
                    guard or similar/

                           >
                           > /we distinctly wanted to be able to tag
                    nova compute hosts with those
                           > new traits so we could require that vms
                    that request/

                           >
                           > /a host with uefi secure boot enabled and
                    a hardware root of
                           trust are
                           > scheduled only to those nodes. /

                           >
                           > / /
                           >
                           > /There are many other examples that effect
                    both vms and bare
                           metal such
                           > as, ecc/interleaved memory, cluster on die, /

                           >
                           > /l3 cache code and data prioritization,
                    vt-d/vt-c, HPET, Hyper
                           > threading, power states … all of these
                    feature may be present on the
                           > platform/

                           >
                           > /but I also need to know if they are
                    turned on. Ruling out state in
                           > traits means all of this logic will
                    eventually get pushed to scheduler
                           > filters/

                           >
                           > /which will be suboptimal long term as
                    more state is tracked.
                           Software
                           > defined infrastructure may be the future
                    but hardware defined
                           software/

                           >
                           > /is sadly the present…/
                           >
                           > / /
                           >
                           > /I do however think there should be a
                    sperateion between asking for a
                           > host that provides x with a trait and
                    asking for x to be
                           configure via/

                           >
                           > /A trait. The trait securebootenabled
                    should never result in the
                           > feature being enabled It should just find a
                    host with it on. If
                           you want/

                           >
                           > /To request it to be turned on you would
                    request a host with
                           > securebootcapable as a trait and have a
                    flavor extra spec or image
                           > property to request/

                           >
                           > /Ironic to enabled it.  these are two very
                    different request and
                           should
                           > not be treated the same. /

                           >
                           >
                           >
                           >
                           >
                           > Lemme know!
                           >
                           > -jay
                           >
                           >
                           >
                           > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur"
                    <dtantsur@redhat.com dtantsur@redhat.com
                    <mailto:dtantsur@redhat.com
                    dtantsur@redhat.com>
                           > <mailto:dtantsur@redhat.com
                    dtantsur@redhat.com
                    <mailto:dtantsur@redhat.com
                    dtantsur@redhat.com>>> wrote:
                           >
                           >     Hi Jay!
                           >
                           >     I appreciate your comments, but I think
                    you're approaching the
                           >     problem from purely VM point of view.
                    Things simply don't work the
                           >     same way in bare metal, at least not if
                    we want to provide the same
                           >     user experience.
                           >
                           >
                           >
                           >     On Sun, Oct 22, 2017 at 2:25 PM, Jay
                    Pipes <jaypipes@gmail.com
                    jaypipes@gmail.com
                    <mailto:jaypipes@gmail.com jaypipes@gmail.com>
                           >     <mailto:jaypipes@gmail.com
                    jaypipes@gmail.com
                    <mailto:jaypipes@gmail.com
                    jaypipes@gmail.com>>> wrote:
                           >
                           >         Sorry for delay, took a week off
                    before starting a new job.
                           >         Comments inline.
                           >
                           >         On 10/16/2017 12:24 PM, Dmitry
                    Tantsur wrote:
                           >
                           >             Hi all,
                           >
                           >             I promised John to dump my
                    thoughts on traits to the
                           ML, so
                           >             here we go :)
                           >
                           >             I see two roles of traits (or
                    kinds of traits) for
                           bare metal:
                           >             1. traits that say what the
                    node can do already (e.g. "the
                           >             node is
                           >             doing UEFI boot")
                           >             2. traits that say what the
                    node can be configured to do
                           >             (e.g. "the node can
                           >             boot in UEFI mode")
                           >
                           >
                           >         There's only one role for traits.
                    #2 above. #1 is state
                           >         information. Traits are not for
                    state information. Traits are
                           >         only for communicating capabilities
                    of a resource provider
                           >         (baremetal node).
                           >
                           >
                           >
                           >     These are not different, that's what
                    I'm talking about here. No
                           >     users care about the difference between
                    "this node was put in UEFI
                           >     mode by an operator in advance", "this
                    node was put in UEFI
                           mode by
                           >     an ironic driver on demand" and "this
                    node is always in UEFI mode,
                           >     because it's AARCH64 and it does not
                    have BIOS". These situation
                           >     produce the same result (the node is
                    booted in UEFI mode), and
                           thus
                           >     it's up to ironic to hide this difference.
                           >
                           >
                           >
                           >     My suggestion with traits is one way to
                    do it, I'm not sure
                           what you
                           >     suggest though.
                           >
                           >
                           >
                           >
                           >         For example, let's say we add the
                    following to the os-traits
                           >         library [1]
                           >
                           >         * STORAGERAID0
                           >         * STORAGERAID1
                           >         * STORAGERAID5
                           >         * STORAGERAID6
                           >         * STORAGERAID10
                           >
                           >         The Ironic administrator would add
                    all RAID-related traits to
                           >         the baremetal nodes that had the
                    capability of
                           supporting that
                           >         particular RAID setup [2]
                           >
                           >         When provisioned, the baremetal
                    node would either have RAID
                           >         configured in a certain level or
                    not configured at all.
                           >
                           >
                           >         A very important note: the
                    Placement API and Nova
                           scheduler (or
                           >         future Ironic scheduler) doesn't
                    care about this. At all.
                           I know
                           >         it sounds like I'm being callous,
                    but I'm not. Placement and
                           >         scheduling doesn't care about the
                    state of things. It only
                           cares
                           >         about the capabilities of target
                    destinations. That's it.
                           >
                           >
                           >
                           >     Yes, because VMs always start with a
                    clean state, and
                           hypervisor is
                           >     there to ensure that. We don't have
                    this luxury in ironic :) E.g.
                           >     our SNMP driver is not even aware of
                    boot modes (or RAID, or BIOS
                           >     configuration), which does not mean
                    that a node using it cannot be
                           >     in UEFI mode (have a RAID or BIOS
                    pre-configured, etc, etc).
                           >
                           >
                           >
                           >
                           >
                           >             This seems confusing, but it's
                    actually very useful.
                           Say, I
                           >             have a flavor that
                           >             requests UEFI boot via a trait.
                    It will match both the
                           nodes
                           >             that are already in
                           >             UEFI mode, as well as nodes
                    that can be put in UEFI mode.
                           >
                           >
                           >         No :) It will only match nodes that
                    have the UEFI capability.
                           >         The set of providers that have the
                    ability to be booted
                           via UEFI
                           >         is always a superset of the set
                    of providers that have been
                           >         booted via UEFI
. Placement and
                    scheduling decisions only care
                           >         about that superset -- the
                    providers with a particular
                           capability.
                           >
                           >
                           >
                           >     Well, no, it will. Again, you're purely
                    basing on the VM idea,
                           where
                           >     a VM is always put in UEFI mode, no
                    matter how the hypervisor
                           >     looks like. It is simply not the case
                    for us. You have to care
                           what
                           >     state the node is, because many drivers
                    cannot change this state.
                           >
                           >
                           >
                           >
                           >
                           >             This idea goes further with
                    deploy templates (new concept
                           >             we've been thinking
                           >             about). A flavor can request
                    something like CUSTOMRAID5,
                           >             and it will match the
                           >             nodes that already have RAID 5,
                    or, more
                           interestingly, the
                           >             nodes on which we
                           >             can build RAID 5 before
                    deployment. The UEFI example above
                           >             can be treated in a
                           >             similar way.
                           >
                           >             This ends up with two sources
                    of knowledge about traits in
                           >             ironic:
                           >             1. Operators setting something
                    they know about hardware
                           >             ("this node is in UEFI
                           >             mode"),
                           >             2. Ironic drivers reporting
                    something they
                           >                2.1. know about hardware
                    ("this node is in UEFI mode" -
                           >             again)
                           >                2.2. can do about hardware
                    ("I can put this node in
                           UEFI
                           >             mode")
                           >
                           >
                           >         You're correct that both pieces of
                    information are important.
                           >         However, only the "can do about
                    hardware" part is relevant to
                           >         Placement and Nova.
                           >
                           >             For case #1 we are planning on
                    a new CRUD API to set/unset
                           >             traits for a node.
                           >
                           >
                           >         I would strongly advise against
                    this. Traits are not for
                           state
                           >         information.
                           >
                           >         Instead, consider having a DB (or
                    JSON) schema that lists
                           state
                           >         information in fields that are
                    explicitly for that state
                           >         information.
                           >
                           >         For example, a schema that looks
                    like this:
                           >
                           >         {
                           >           "boot": {
                           >             "mode": ,
                           >             "params":
                           >           },
                           >           "disk": {
                           >             "raid": {
                           >               "level": ,
                           >               "controller": ,
                           >               "driver": ,
                           >               "params":
                           >             },  ...
                           >           },
                           >           "network": {
                           >             ...
                           >           }
                           >         }
                           >
                           >         etc, etc.
                           >
                           >         Don't use trait strings to
                    represent state information.
                           >
                           >
                           >
                           >     I don't see an alternative proposal
                    that will satisfy what we have
                           >     to solve.
                           >
                           >
                           >
                           >
                           >         Best,
                           >         -jay
                           >
                           >             Case #2 is more interesting. We
                    have two options, I think:
                           >
                           >             a) Operators still set traits
                    on nodes, drivers are simply
                           >             validating them. E.g.
                           >             an operators sets
                    CUSTOMRAID5, and the node's RAID
                           >             interface checks if it is
                           >             possible to do. The downside is
                    obvious - with a lot of
                           >             deploy templates
                           >             available it can be a lot of
                    manual work.
                           >
                           >             b) Drivers report the traits,
                    and they get somehow
                           added to
                           >             the traits provided
                           >             by an operator. Technically,
                    there are sub-cases again:
                           >                b.1) The new traits API
                    returns a union of
                           >             operator-provided and
                           >             driver-provided traits
                           >                b.2) The new traits API
                    returns only operator-provided
                           >             traits; driver-provided
                           >             traits are returned e.g. via a
                    new field
                           >             (node.driver_traits). Then nova
                    will
                           >             have to merge the lists itself.
                           >
                           >             My personal favorite is the
                    last option: I'd like a clear
                           >             distinction between
                           >             different "sources" of traits,
                    but I'd also like to reduce
                           >             manual work for
                           >             operators.
                           >
                           >             A valid counter-argument is:
                    what if an operator wants to
                           >             override a
                           >             driver-provided trait? E.g. a
                    node can do RAID 5, but I
                           >             don't want this
                           >             particular node to do it for
                    any reason. I'm not sure if
                           >             it's a valid case, and
                           >             what to do about it.
                           >
                           >             Let me know what you think.
                           >
                           >             Dmitry
                           >
                           >
                           >         [1]
                    http://git.openstack.org/cgit/openstack/os-traits/tree/
                   
                    >                     >
                           >         [2] Based on how many attached
                    disks the node had, the
                           presence
                           >         and abilities of a hardware RAID
                    controller, etc
                           >
                           >
                           >
                           >

                           >         OpenStack Development Mailing List
                    (not for usage questions)
                           >         Unsubscribe:
                           >
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

                
                

                           >

                
                
                

>
                           >

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                
                

                           >
                           >
                           >
                           >
                           >


                           >     OpenStack Development Mailing List (not
                    for usage questions)
                           >     Unsubscribe:
                           >
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                

                           >

                
                
                

>
                           >

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                
                

                           >
                           >
                           >
                           >


                           > OpenStack Development Mailing List (not for
                    usage questions)
                           > Unsubscribe:
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                

                           >

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                
                

                           >

                


                           OpenStack Development Mailing List (not for
                    usage questions)
                           Unsubscribe:
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                
                
                


                    OpenStack Development Mailing List (not for usage
                    questions)
                    Unsubscribe:
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

            


                OpenStack Development Mailing List (not for usage questions)
                Unsubscribe:
                OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

                http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

    __________________________________________________________________________
    OpenStack Development Mailing List (not for usage questions)
    Unsubscribe:
    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
   
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
   

--
--
-- Dmitry Tantsur
--


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 30, 2017 by Dmitry_Tantsur (18,080 points)   2 3 4
0 votes

It seems that the new time works for the most of key people, so let's move it to
tomorrow (Tue), the same time, the same bluejeans.

Apologies to those who won't be able to attend, and sorry for the late notice.

On 10/30/2017 03:13 PM, Jay Pipes wrote:
I'd prefer to have you on the call, Dima. How about we push it back to tomorrow
at the same time?

Can everyone make it then?

-jay

On 10/30/2017 10:11 AM, Dmitry Tantsur wrote:

Aaaand sorry again, but due to sudden errands I won't be able to attend.
Please feel free to use my bluejeans room anyway. I think my position on
traits is more or less clear from previous discussions with John, Sam and Eric.

2017-10-24 18:07 GMT+02:00 Dmitry Tantsur <dtantsur@redhat.com
dtantsur@redhat.com>:

    Sigh, sorry. I forgot that we're moving back to winter time this
    weekend. I think the time is 3pm UTC then. It seems to be 11am
    eastern US:

https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et


https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et.

    On 10/24/2017 06:00 PM, Dmitry Tantsur wrote:

        And the winner is Mon, 30 Oct, 2pm UTC!

        The bluejeans ID is https://bluejeans.com/757528759
        https://bluejeans.com/757528759
        (works without plugins in recent FF and Chrome; if it asks to
        install an app, ignore it and look for a link saying "join with
        browser")

        On 10/23/2017 05:02 PM, Dmitry Tantsur wrote:

            Hi all!

            I'd like to invite you to the discussion of the way to
            implement traits in
            ironic and the ironic virt driver. Please vote for the time at
            https://doodle.com/poll/ts43k98kkvniv8uz
            https://doodle.com/poll/ts43k98kkvniv8uz. Please vote by
            EOD tomorrow.

            Note that it's going to be a technical discussion - please
            make sure you
            understand what traits are and why ironic cares about them.
            See below for more
            context.

            We'll probably use my bluejeans account, as it works without
            plugins in modern
            browsers. I'll post a meeting ID when we pick the date.

            On 10/23/2017 04:09 PM, Eric Fried wrote:

                We discussed this a little bit further in IRC [1].
                We're all in
                agreement, but it's worth being precise on a couple of
                points:

                * We're distinguishing between a "feature" and the
                "trait" that
                represents it in placement.  For the sake of this
                discussion, a
                "feature" can (maybe) be switched on or off, but a
                "trait" can either be
                present or absent on a RP.
                * It matters who can turn a feature on/off.
                     * If it can be done by virt at spawn time, then it
                makes sense to have
                the trait on the RP, and you can switch the feature
                on/off via a
                separate extra_spec.
                     * But if it's e.g. an admin action, and spawn has
                no control, then the
                trait needs to be *added* whenever the feature is *on*,
                and *removed*
                whenever the feature is *off*.

                [1]

http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13


                On 10/23/2017 08:15 AM, Sylvain Bauza wrote:

                    On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried
                    <openstack@fried.cc
                    <mailto:openstack@fried.cc
                    openstack@fried.cc>> wrote:

                           I agree with Sean.  In general terms:

                           * A resource provider should be marked with a
                    trait if that feature
                             * Can be turned on or off (whether it's
                    currently on or not); or
                             * Is always on and can't ever be turned off.

                    No, traits are not boolean. If a resource provider
                    stops providing a
                    capability, then the existing related trait should
                    just be removed,
                    that's it.
                    If you see a trait, that's just means that the
                    related capability for
                    the Resource Provider is supported, that's it too.

                    MHO.

                    -Sylvain

                           * A consumer wanting that feature present
                    (doesn't matter whether it's
                           on or off) should specify it as a required
                    trait.
                           * A consumer wanting that feature present and
                    turned on should
                             * Specify it as a required trait; AND
                             * Indicate that it be turned on via some
                    other mechanism (e.g. a
                           separate extra_spec).

                           I believe this satisfies Dmitry's (Ironic's)
                    needs, but also Jay's drive
                           for placement purity.

                           Please invite me to the hangout or whatever.

                           Thanks,
                           Eric

                           On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
                           >
                           >
                           >
                           >
                           > From:Jay Pipes [mailto:jaypipes@gmail.com
                    jaypipes@gmail.com
                           <mailto:jaypipes@gmail.com
                    jaypipes@gmail.com>]
                           > Sent: Monday, October 23, 2017 12:20 PM
                           > To: OpenStack Development Mailing List
                           <openstack-dev@lists.openstack.org
                    openstack-dev@lists.openstack.org
                           <mailto:openstack-dev@lists.openstack.org
                    openstack-dev@lists.openstack.org>>
                           > Subject: Re: [openstack-dev] [ironic]
                    ironic and traits
                           >
                           >
                           >
                           > Writing from my phone... May I ask that
                    before you proceed with any plan
                           > that uses traits for state information that
                    we have a hangout or
                           > videoconference to discuss this?
                    Unfortunately today and tomorrow I'm
                           > not able to do a hangout but I can do one
                    on Wednesday any time of the day.
                           >
                           >
                           >
                           > /[Mooney, Sean K] on the uefi boot topic I
                    did bring up at the
                           ptg that
                           > we wanted to standardizes tratis for
                    “verified boot” /

                           >
                           > /that included a trait for uefi secure
                    boot enabled and to
                           indicated a
                           > hardware root of trust, e.g. intel boot
                    guard or similar/

                           >
                           > /we distinctly wanted to be able to tag
                    nova compute hosts with those
                           > new traits so we could require that vms
                    that request/

                           >
                           > /a host with uefi secure boot enabled and
                    a hardware root of
                           trust are
                           > scheduled only to those nodes. /

                           >
                           > / /
                           >
                           > /There are many other examples that effect
                    both vms and bare
                           metal such
                           > as, ecc/interleaved memory, cluster on die, /

                           >
                           > /l3 cache code and data prioritization,
                    vt-d/vt-c, HPET, Hyper
                           > threading, power states … all of these
                    feature may be present on the
                           > platform/

                           >
                           > /but I also need to know if they are
                    turned on. Ruling out state in
                           > traits means all of this logic will
                    eventually get pushed to scheduler
                           > filters/

                           >
                           > /which will be suboptimal long term as
                    more state is tracked.
                           Software
                           > defined infrastructure may be the future
                    but hardware defined
                           software/

                           >
                           > /is sadly the present…/
                           >
                           > / /
                           >
                           > /I do however think there should be a
                    sperateion between asking for a
                           > host that provides x with a trait and
                    asking for x to be
                           configure via/

                           >
                           > /A trait. The trait securebootenabled
                    should never result in the
                           > feature being enabled It should just find a
                    host with it on. If
                           you want/

                           >
                           > /To request it to be turned on you would
                    request a host with
                           > securebootcapable as a trait and have a
                    flavor extra spec or image
                           > property to request/

                           >
                           > /Ironic to enabled it.  these are two very
                    different request and
                           should
                           > not be treated the same. /

                           >
                           >
                           >
                           >
                           >
                           > Lemme know!
                           >
                           > -jay
                           >
                           >
                           >
                           > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur"
                    <dtantsur@redhat.com dtantsur@redhat.com
                    <mailto:dtantsur@redhat.com
                    dtantsur@redhat.com>
                           > <mailto:dtantsur@redhat.com
                    dtantsur@redhat.com
                    <mailto:dtantsur@redhat.com
                    dtantsur@redhat.com>>> wrote:
                           >
                           >     Hi Jay!
                           >
                           >     I appreciate your comments, but I think
                    you're approaching the
                           >     problem from purely VM point of view.
                    Things simply don't work the
                           >     same way in bare metal, at least not if
                    we want to provide the same
                           >     user experience.
                           >
                           >
                           >
                           >     On Sun, Oct 22, 2017 at 2:25 PM, Jay
                    Pipes <jaypipes@gmail.com
                    jaypipes@gmail.com
                    <mailto:jaypipes@gmail.com jaypipes@gmail.com>
                           >     <mailto:jaypipes@gmail.com
                    jaypipes@gmail.com
                    <mailto:jaypipes@gmail.com
                    jaypipes@gmail.com>>> wrote:
                           >
                           >         Sorry for delay, took a week off
                    before starting a new job.
                           >         Comments inline.
                           >
                           >         On 10/16/2017 12:24 PM, Dmitry
                    Tantsur wrote:
                           >
                           >             Hi all,
                           >
                           >             I promised John to dump my
                    thoughts on traits to the
                           ML, so
                           >             here we go :)
                           >
                           >             I see two roles of traits (or
                    kinds of traits) for
                           bare metal:
                           >             1. traits that say what the
                    node can do already (e.g. "the
                           >             node is
                           >             doing UEFI boot")
                           >             2. traits that say what the
                    node can be configured to do
                           >             (e.g. "the node can
                           >             boot in UEFI mode")
                           >
                           >
                           >         There's only one role for traits.
                    #2 above. #1 is state
                           >         information. Traits are not for
                    state information. Traits are
                           >         only for communicating capabilities
                    of a resource provider
                           >         (baremetal node).
                           >
                           >
                           >
                           >     These are not different, that's what
                    I'm talking about here. No
                           >     users care about the difference between
                    "this node was put in UEFI
                           >     mode by an operator in advance", "this
                    node was put in UEFI
                           mode by
                           >     an ironic driver on demand" and "this
                    node is always in UEFI mode,
                           >     because it's AARCH64 and it does not
                    have BIOS". These situation
                           >     produce the same result (the node is
                    booted in UEFI mode), and
                           thus
                           >     it's up to ironic to hide this difference.
                           >
                           >
                           >
                           >     My suggestion with traits is one way to
                    do it, I'm not sure
                           what you
                           >     suggest though.
                           >
                           >
                           >
                           >
                           >         For example, let's say we add the
                    following to the os-traits
                           >         library [1]
                           >
                           >         * STORAGERAID0
                           >         * STORAGERAID1
                           >         * STORAGERAID5
                           >         * STORAGERAID6
                           >         * STORAGERAID10
                           >
                           >         The Ironic administrator would add
                    all RAID-related traits to
                           >         the baremetal nodes that had the
                    capability of
                           supporting that
                           >         particular RAID setup [2]
                           >
                           >         When provisioned, the baremetal
                    node would either have RAID
                           >         configured in a certain level or
                    not configured at all.
                           >
                           >
                           >         A very important note: the
                    Placement API and Nova
                           scheduler (or
                           >         future Ironic scheduler) doesn't
                    care about this. At all.
                           I know
                           >         it sounds like I'm being callous,
                    but I'm not. Placement and
                           >         scheduling doesn't care about the
                    state of things. It only
                           cares
                           >         about the capabilities of target
                    destinations. That's it.
                           >
                           >
                           >
                           >     Yes, because VMs always start with a
                    clean state, and
                           hypervisor is
                           >     there to ensure that. We don't have
                    this luxury in ironic :) E.g.
                           >     our SNMP driver is not even aware of
                    boot modes (or RAID, or BIOS
                           >     configuration), which does not mean
                    that a node using it cannot be
                           >     in UEFI mode (have a RAID or BIOS
                    pre-configured, etc, etc).
                           >
                           >
                           >
                           >
                           >
                           >             This seems confusing, but it's
                    actually very useful.
                           Say, I
                           >             have a flavor that
                           >             requests UEFI boot via a trait.
                    It will match both the
                           nodes
                           >             that are already in
                           >             UEFI mode, as well as nodes
                    that can be put in UEFI mode.
                           >
                           >
                           >         No :) It will only match nodes that
                    have the UEFI capability.
                           >         The set of providers that have the
                    ability to be booted
                           via UEFI
                           >         is always a superset of the set
                    of providers that have been
                           >         booted via UEFI
. Placement and
                    scheduling decisions only care
                           >         about that superset -- the
                    providers with a particular
                           capability.
                           >
                           >
                           >
                           >     Well, no, it will. Again, you're purely
                    basing on the VM idea,
                           where
                           >     a VM is always put in UEFI mode, no
                    matter how the hypervisor
                           >     looks like. It is simply not the case
                    for us. You have to care
                           what
                           >     state the node is, because many drivers
                    cannot change this state.
                           >
                           >
                           >
                           >
                           >
                           >             This idea goes further with
                    deploy templates (new concept
                           >             we've been thinking
                           >             about). A flavor can request
                    something like CUSTOMRAID5,
                           >             and it will match the
                           >             nodes that already have RAID 5,
                    or, more
                           interestingly, the
                           >             nodes on which we
                           >             can build RAID 5 before
                    deployment. The UEFI example above
                           >             can be treated in a
                           >             similar way.
                           >
                           >             This ends up with two sources
                    of knowledge about traits in
                           >             ironic:
                           >             1. Operators setting something
                    they know about hardware
                           >             ("this node is in UEFI
                           >             mode"),
                           >             2. Ironic drivers reporting
                    something they
                           >                2.1. know about hardware
                    ("this node is in UEFI mode" -
                           >             again)
                           >                2.2. can do about hardware
                    ("I can put this node in
                           UEFI
                           >             mode")
                           >
                           >
                           >         You're correct that both pieces of
                    information are important.
                           >         However, only the "can do about
                    hardware" part is relevant to
                           >         Placement and Nova.
                           >
                           >             For case #1 we are planning on
                    a new CRUD API to set/unset
                           >             traits for a node.
                           >
                           >
                           >         I would strongly advise against
                    this. Traits are not for
                           state
                           >         information.
                           >
                           >         Instead, consider having a DB (or
                    JSON) schema that lists
                           state
                           >         information in fields that are
                    explicitly for that state
                           >         information.
                           >
                           >         For example, a schema that looks
                    like this:
                           >
                           >         {
                           >           "boot": {
                           >             "mode": ,
                           >             "params":
                           >           },
                           >           "disk": {
                           >             "raid": {
                           >               "level": ,
                           >               "controller": ,
                           >               "driver": ,
                           >               "params":
                           >             },  ...
                           >           },
                           >           "network": {
                           >             ...
                           >           }
                           >         }
                           >
                           >         etc, etc.
                           >
                           >         Don't use trait strings to
                    represent state information.
                           >
                           >
                           >
                           >     I don't see an alternative proposal
                    that will satisfy what we have
                           >     to solve.
                           >
                           >
                           >
                           >
                           >         Best,
                           >         -jay
                           >
                           >             Case #2 is more interesting. We
                    have two options, I think:
                           >
                           >             a) Operators still set traits
                    on nodes, drivers are simply
                           >             validating them. E.g.
                           >             an operators sets
                    CUSTOMRAID5, and the node's RAID
                           >             interface checks if it is
                           >             possible to do. The downside is
                    obvious - with a lot of
                           >             deploy templates
                           >             available it can be a lot of
                    manual work.
                           >
                           >             b) Drivers report the traits,
                    and they get somehow
                           added to
                           >             the traits provided
                           >             by an operator. Technically,
                    there are sub-cases again:
                           >                b.1) The new traits API
                    returns a union of
                           >             operator-provided and
                           >             driver-provided traits
                           >                b.2) The new traits API
                    returns only operator-provided
                           >             traits; driver-provided
                           >             traits are returned e.g. via a
                    new field
                           >             (node.driver_traits). Then nova
                    will
                           >             have to merge the lists itself.
                           >
                           >             My personal favorite is the
                    last option: I'd like a clear
                           >             distinction between
                           >             different "sources" of traits,
                    but I'd also like to reduce
                           >             manual work for
                           >             operators.
                           >
                           >             A valid counter-argument is:
                    what if an operator wants to
                           >             override a
                           >             driver-provided trait? E.g. a
                    node can do RAID 5, but I
                           >             don't want this
                           >             particular node to do it for
                    any reason. I'm not sure if
                           >             it's a valid case, and
                           >             what to do about it.
                           >
                           >             Let me know what you think.
                           >
                           >             Dmitry
                           >
                           >
                           >         [1]
                    http://git.openstack.org/cgit/openstack/os-traits/tree/
                   
                    >                     >
                           >         [2] Based on how many attached
                    disks the node had, the
                           presence
                           >         and abilities of a hardware RAID
                    controller, etc
                           >
                           >
                           >
                           >

                           >         OpenStack Development Mailing List
                    (not for usage questions)
                           >         Unsubscribe:
                           >
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

                
                

                           >

                
                
                

>
                           >

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                
                

                           >
                           >
                           >
                           >
                           >


                           >     OpenStack Development Mailing List (not
                    for usage questions)
                           >     Unsubscribe:
                           >
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                

                           >

                
                
                

>
                           >

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                
                

                           >
                           >
                           >
                           >


                           > OpenStack Development Mailing List (not for
                    usage questions)
                           > Unsubscribe:
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                

                           >

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                
                

                           >

                


                           OpenStack Development Mailing List (not for
                    usage questions)
                           Unsubscribe:
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                
                

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                
                
                


                    OpenStack Development Mailing List (not for usage
                    questions)
                    Unsubscribe:
                    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
                

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

            


                OpenStack Development Mailing List (not for usage questions)
                Unsubscribe:
                OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

                http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

    __________________________________________________________________________
    OpenStack Development Mailing List (not for usage questions)
    Unsubscribe:
    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
   
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
   

--
--
-- Dmitry Tantsur
--


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 30, 2017 by Dmitry_Tantsur (18,080 points)   2 3 4
0 votes

On 10/30/2017 9:32 AM, Dmitry Tantsur wrote:
the same bluejeans.

Forever in bluejeans?

--

Thanks,

Matt


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 30, 2017 by mriedemos_at_gmail.c (15,720 points)   2 4 4
0 votes

Hi all,

The recording of the call is https://bluejeans.com/s/K3wZZ

On 10/30/2017 03:32 PM, Dmitry Tantsur wrote:
It seems that the new time works for the most of key people, so let's move it to
tomorrow (Tue), the same time, the same bluejeans.

Apologies to those who won't be able to attend, and sorry for the late notice.

On 10/30/2017 03:13 PM, Jay Pipes wrote:

I'd prefer to have you on the call, Dima. How about we push it back to
tomorrow at the same time?

Can everyone make it then?

-jay

On 10/30/2017 10:11 AM, Dmitry Tantsur wrote:

Aaaand sorry again, but due to sudden errands I won't be able to attend.
Please feel free to use my bluejeans room anyway. I think my position on
traits is more or less clear from previous discussions with John, Sam and Eric.

2017-10-24 18:07 GMT+02:00 Dmitry Tantsur <dtantsur@redhat.com
dtantsur@redhat.com>:

    Sigh, sorry. I forgot that we're moving back to winter time this
    weekend. I think the time is 3pm UTC then. It seems to be 11am
    eastern US:
https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et

https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et.

    On 10/24/2017 06:00 PM, Dmitry Tantsur wrote:

        And the winner is Mon, 30 Oct, 2pm UTC!

        The bluejeans ID is https://bluejeans.com/757528759
        https://bluejeans.com/757528759
        (works without plugins in recent FF and Chrome; if it asks to
        install an app, ignore it and look for a link saying "join with
        browser")

        On 10/23/2017 05:02 PM, Dmitry Tantsur wrote:

            Hi all!

            I'd like to invite you to the discussion of the way to
            implement traits in
            ironic and the ironic virt driver. Please vote for the time at
            https://doodle.com/poll/ts43k98kkvniv8uz
            https://doodle.com/poll/ts43k98kkvniv8uz. Please vote by
            EOD tomorrow.

            Note that it's going to be a technical discussion - please
            make sure you
            understand what traits are and why ironic cares about them.
            See below for more
            context.

            We'll probably use my bluejeans account, as it works without
            plugins in modern
            browsers. I'll post a meeting ID when we pick the date.

            On 10/23/2017 04:09 PM, Eric Fried wrote:

                We discussed this a little bit further in IRC [1].
                We're all in
                agreement, but it's worth being precise on a couple of
                points:

                * We're distinguishing between a "feature" and the
                "trait" that
                represents it in placement.  For the sake of this
                discussion, a
                "feature" can (maybe) be switched on or off, but a
                "trait" can either be
                present or absent on a RP.
                * It matters who can turn a feature on/off.
                     * If it can be done by virt at spawn time, then it
                makes sense to have
                the trait on the RP, and you can switch the feature
                on/off via a
                separate extra_spec.
                     * But if it's e.g. an admin action, and spawn has
                no control, then the
                trait needs to be *added* whenever the feature is *on*,
                and *removed*
                whenever the feature is *off*.

                [1]
http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13

                On 10/23/2017 08:15 AM, Sylvain Bauza wrote:

                    On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried
                    <openstack@fried.cc
                    <mailto:openstack@fried.cc
                    openstack@fried.cc>> wrote:

                           I agree with Sean.  In general terms:

                           * A resource provider should be marked with a
                    trait if that feature
                             * Can be turned on or off (whether it's
                    currently on or not); or
                             * Is always on and can't ever be turned off.

                    No, traits are not boolean. If a resource provider
                    stops providing a
                    capability, then the existing related trait should
                    just be removed,
                    that's it.
                    If you see a trait, that's just means that the
                    related capability for
                    the Resource Provider is supported, that's it too.

                    MHO.

                    -Sylvain

                           * A consumer wanting that feature present
                    (doesn't matter whether it's
                           on or off) should specify it as a required
                    trait.
                           * A consumer wanting that feature present and
                    turned on should
                             * Specify it as a required trait; AND
                             * Indicate that it be turned on via some
                    other mechanism (e.g. a
                           separate extra_spec).

                           I believe this satisfies Dmitry's (Ironic's)
                    needs, but also Jay's drive
                           for placement purity.

                           Please invite me to the hangout or whatever.

                           Thanks,
                           Eric

                           On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
                           >
                           >
                           >
                           >
                           > From:Jay Pipes [mailto:jaypipes@gmail.com
                    jaypipes@gmail.com
                           <mailto:jaypipes@gmail.com
                    jaypipes@gmail.com>]
                           > Sent: Monday, October 23, 2017 12:20 PM
                           > To: OpenStack Development Mailing List
                           <openstack-dev@lists.openstack.org
                    openstack-dev@lists.openstack.org
                           <mailto:openstack-dev@lists.openstack.org
                    openstack-dev@lists.openstack.org>>
                           > Subject: Re: [openstack-dev] [ironic]
                    ironic and traits
                           >
                           >
                           >
                           > Writing from my phone... May I ask that
                    before you proceed with any plan
                           > that uses traits for state information that
                    we have a hangout or
                           > videoconference to discuss this?
                    Unfortunately today and tomorrow I'm
                           > not able to do a hangout but I can do one
                    on Wednesday any time of the day.
                           >
                           >
                           >
                           > /[Mooney, Sean K] on the uefi boot topic I
                    did bring up at the
                           ptg that
                           > we wanted to standardizes tratis for
                    “verified boot” /

                           >
                           > /that included a trait for uefi secure
                    boot enabled and to
                           indicated a
                           > hardware root of trust, e.g. intel boot
                    guard or similar/

                           >
                           > /we distinctly wanted to be able to tag
                    nova compute hosts with those
                           > new traits so we could require that vms
                    that request/

                           >
                           > /a host with uefi secure boot enabled and
                    a hardware root of
                           trust are
                           > scheduled only to those nodes. /

                           >
                           > / /
                           >
                           > /There are many other examples that effect
                    both vms and bare
                           metal such
                           > as, ecc/interleaved memory, cluster on die, /

                           >
                           > /l3 cache code and data prioritization,
                    vt-d/vt-c, HPET, Hyper
                           > threading, power states … all of these
                    feature may be present on the
                           > platform/

                           >
                           > /but I also need to know if they are
                    turned on. Ruling out state in
                           > traits means all of this logic will
                    eventually get pushed to scheduler
                           > filters/

                           >
                           > /which will be suboptimal long term as
                    more state is tracked.
                           Software
                           > defined infrastructure may be the future
                    but hardware defined
                           software/

                           >
                           > /is sadly the present…/
                           >
                           > / /
                           >
                           > /I do however think there should be a
                    sperateion between asking for a
                           > host that provides x with a trait and
                    asking for x to be
                           configure via/

                           >
                           > /A trait. The trait securebootenabled
                    should never result in the
                           > feature being enabled It should just find a
                    host with it on. If
                           you want/

                           >
                           > /To request it to be turned on you would
                    request a host with
                           > securebootcapable as a trait and have a
                    flavor extra spec or image
                           > property to request/

                           >
                           > /Ironic to enabled it.  these are two very
                    different request and
                           should
                           > not be treated the same. /

                           >
                           >
                           >
                           >
                           >
                           > Lemme know!
                           >
                           > -jay
                           >
                           >
                           >
                           > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur"
                    <dtantsur@redhat.com dtantsur@redhat.com
                    <mailto:dtantsur@redhat.com
                    dtantsur@redhat.com>
                           > <mailto:dtantsur@redhat.com
                    dtantsur@redhat.com
                    <mailto:dtantsur@redhat.com
                    dtantsur@redhat.com>>> wrote:
                           >
                           >     Hi Jay!
                           >
                           >     I appreciate your comments, but I think
                    you're approaching the
                           >     problem from purely VM point of view.
                    Things simply don't work the
                           >     same way in bare metal, at least not if
                    we want to provide the same
                           >     user experience.
                           >
                           >
                           >
                           >     On Sun, Oct 22, 2017 at 2:25 PM, Jay
                    Pipes <jaypipes@gmail.com
                    jaypipes@gmail.com
                    <mailto:jaypipes@gmail.com jaypipes@gmail.com>
                           >     <mailto:jaypipes@gmail.com
                    jaypipes@gmail.com
                    <mailto:jaypipes@gmail.com
                    jaypipes@gmail.com>>> wrote:
                           >
                           >         Sorry for delay, took a week off
                    before starting a new job.
                           >         Comments inline.
                           >
                           >         On 10/16/2017 12:24 PM, Dmitry
                    Tantsur wrote:
                           >
                           >             Hi all,
                           >
                           >             I promised John to dump my
                    thoughts on traits to the
                           ML, so
                           >             here we go :)
                           >
                           >             I see two roles of traits (or
                    kinds of traits) for
                           bare metal:
                           >             1. traits that say what the
                    node can do already (e.g. "the
                           >             node is
                           >             doing UEFI boot")
                           >             2. traits that say what the
                    node can be configured to do
                           >             (e.g. "the node can
                           >             boot in UEFI mode")
                           >
                           >
                           >         There's only one role for traits.
                    #2 above. #1 is state
                           >         information. Traits are not for
                    state information. Traits are
                           >         only for communicating capabilities
                    of a resource provider
                           >         (baremetal node).
                           >
                           >
                           >
                           >     These are not different, that's what
                    I'm talking about here. No
                           >     users care about the difference between
                    "this node was put in UEFI
                           >     mode by an operator in advance", "this
                    node was put in UEFI
                           mode by
                           >     an ironic driver on demand" and "this
                    node is always in UEFI mode,
                           >     because it's AARCH64 and it does not
                    have BIOS". These situation
                           >     produce the same result (the node is
                    booted in UEFI mode), and
                           thus
                           >     it's up to ironic to hide this difference.
                           >
                           >
                           >
                           >     My suggestion with traits is one way to
                    do it, I'm not sure
                           what you
                           >     suggest though.
                           >
                           >
                           >
                           >
                           >         For example, let's say we add the
                    following to the os-traits
                           >         library [1]
                           >
                           >         * STORAGERAID0
                           >         * STORAGERAID1
                           >         * STORAGERAID5
                           >         * STORAGERAID6
                           >         * STORAGERAID10
                           >
                           >         The Ironic administrator would add
                    all RAID-related traits to
                           >         the baremetal nodes that had the
                    capability of
                           supporting that
                           >         particular RAID setup [2]
                           >
                           >         When provisioned, the baremetal
                    node would either have RAID
                           >         configured in a certain level or
                    not configured at all.
                           >
                           >
                           >         A very important note: the
                    Placement API and Nova
                           scheduler (or
                           >         future Ironic scheduler) doesn't
                    care about this. At all.
                           I know
                           >         it sounds like I'm being callous,
                    but I'm not. Placement and
                           >         scheduling doesn't care about the
                    state of things. It only
                           cares
                           >         about the capabilities of target
                    destinations. That's it.
                           >
                           >
                           >
                           >     Yes, because VMs always start with a
                    clean state, and
                           hypervisor is
                           >     there to ensure that. We don't have
                    this luxury in ironic :) E.g.
                           >     our SNMP driver is not even aware of
                    boot modes (or RAID, or BIOS
                           >     configuration), which does not mean
                    that a node using it cannot be
                           >     in UEFI mode (have a RAID or BIOS
                    pre-configured, etc, etc).
                           >
                           >
                           >
                           >
                           >
                           >             This seems confusing, but it's
                    actually very useful.
                           Say, I
                           >             have a flavor that
                           >             requests UEFI boot via a trait.
                    It will match both the
                           nodes
                           >             that are already in
                           >             UEFI mode, as well as nodes
                    that can be put in UEFI mode.
                           >
                           >
                           >         No :) It will only match nodes that
                    have the UEFI capability.
                           >         The set of providers that have the
                    ability to be booted
                           via UEFI
                           >         is always a superset of the set
                    of providers that have been
                           >         booted via UEFI
. Placement and
                    scheduling decisions only care
                           >         about that superset -- the
                    providers with a particular
                           capability.
                           >
                           >
                           >
                           >     Well, no, it will. Again, you're purely
                    basing on the VM idea,
                           where
                           >     a VM is always put in UEFI mode, no
                    matter how the hypervisor
                           >     looks like. It is simply not the case
                    for us. You have to care
                           what
                           >     state the node is, because many drivers
                    cannot change this state.
                           >
                           >
                           >
                           >
                           >
                           >             This idea goes further with
                    deploy templates (new concept
                           >             we've been thinking
                           >             about). A flavor can request
                    something like CUSTOMRAID5,
                           >             and it will match the
                           >             nodes that already have RAID 5,
                    or, more
                           interestingly, the
                           >             nodes on which we
                           >             can build RAID 5 before
                    deployment. The UEFI example above
                           >             can be treated in a
                           >             similar way.
                           >
                           >             This ends up with two sources
                    of knowledge about traits in
                           >             ironic:
                           >             1. Operators setting something
                    they know about hardware
                           >             ("this node is in UEFI
                           >             mode"),
                           >             2. Ironic drivers reporting
                    something they
                           >                2.1. know about hardware
                    ("this node is in UEFI mode" -
                           >             again)
                           >                2.2. can do about hardware
                    ("I can put this node in
                           UEFI
                           >             mode")
                           >
                           >
                           >         You're correct that both pieces of
                    information are important.
                           >         However, only the "can do about
                    hardware" part is relevant to
                           >         Placement and Nova.
                           >
                           >             For case #1 we are planning on
                    a new CRUD API to set/unset
                           >             traits for a node.
                           >
                           >
                           >         I would strongly advise against
                    this. Traits are not for
                           state
                           >         information.
                           >
                           >         Instead, consider having a DB (or
                    JSON) schema that lists
                           state
                           >         information in fields that are
                    explicitly for that state
                           >         information.
                           >
                           >         For example, a schema that looks
                    like this:
                           >
                           >         {
                           >           "boot": {
                           >             "mode": ,
                           >             "params":
                           >           },
                           >           "disk": {
                           >             "raid": {
                           >               "level": ,
                           >               "controller": ,
                           >               "driver": ,
                           >               "params":
                           >             },  ...
                           >           },
                           >           "network": {
                           >             ...
                           >           }
                           >         }
                           >
                           >         etc, etc.
                           >
                           >         Don't use trait strings to
                    represent state information.
                           >
                           >
                           >
                           >     I don't see an alternative proposal
                    that will satisfy what we have
                           >     to solve.
                           >
                           >
                           >
                           >
                           >         Best,
                           >         -jay
                           >
                           >             Case #2 is more interesting. We
                    have two options, I think:
                           >
                           >             a) Operators still set traits
                    on nodes, drivers are simply
                           >             validating them. E.g.
                           >             an operators sets
                    CUSTOMRAID5, and the node's RAID
                           >             interface checks if it is
                           >             possible to do. The downside is
                    obvious - with a lot of
                           >             deploy templates
                           >             available it can be a lot of
                    manual work.
                           >
                           >             b) Drivers report the traits,
                    and they get somehow
                           added to
                           >             the traits provided
                           >             by an operator. Technically,
                    there are sub-cases again:
                           >                b.1) The new traits API
                    returns a union of
                           >             operator-provided and
                           >             driver-provided traits
                           >                b.2) The new traits API
                    returns only operator-provided
                           >             traits; driver-provided
                           >             traits are returned e.g. via a
                    new field
                           >             (node.driver_traits). Then nova
                    will
                           >             have to merge the lists itself.
                           >
                           >             My personal favorite is the
                    last option: I'd like a clear
                           >             distinction between
                           >             different "sources" of traits,
                    but I'd also like to reduce
                           >             manual work for
                           >             operators.
                           >
                           >             A valid counter-argument is:
                    what if an operator wants to
                           >             override a
                           >             driver-provided trait? E.g. a
                    node can do RAID 5, but I
                           >             don't want this
                           >             particular node to do it for
                    any reason. I'm not sure if
                           >             it's a valid case, and
                           >             what to do about it.
                           >
                           >             Let me know what you think.
                           >
                           >             Dmitry
                           >
                           >
                           >         [1]
                    http://git.openstack.org/cgit/openstack/os-traits/tree/
                   
                    >>                     >
                           >         [2] Based on how many attached
                    disks the node had, the
                           presence
                           >         and abilities of a hardware RAID
                    controller, etc
                           >
                           >
                           >
                           >
                           >         OpenStack Development Mailing List
                    (not for usage questions)
                           >         Unsubscribe:
                           >

OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

>
                           >

>
                           >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                           >
                           >
                           >
                           >
                           >


                           >     OpenStack Development Mailing List (not
                    for usage questions)
                           >     Unsubscribe:
                           >

OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

>
                           >

>
                           >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

                           >
                           >
                           >
                           >


                           > OpenStack Development Mailing List (not for
                    usage questions)
                           > Unsubscribe:

OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

>
                           >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

>
                           >


                           OpenStack Development Mailing List (not for
                    usage questions)
                           Unsubscribe:

OpenStack-dev-request@lists.openstack.org?subject:unsubscribe


                    OpenStack Development Mailing List (not for usage
                    questions)
                    Unsubscribe:

OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


                OpenStack Development Mailing List (not for usage questions)
                Unsubscribe:
                OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
            

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

    __________________________________________________________________________
    OpenStack Development Mailing List (not for usage questions)
    Unsubscribe:
    OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
   
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
   

--
--
-- Dmitry Tantsur
--


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Nov 2, 2017 by Dmitry_Tantsur (18,080 points)   2 3 4
0 votes

On Nov 2, 2017, at 4:17 AM, Dmitry Tantsur dtantsur@redhat.com wrote:

The recording of the call is https://bluejeans.com/s/K3wZZ

Ugh: "To watch the video, you need Adobe Flash Player 10.1 or higher"

-- Ed Leafe


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Nov 2, 2017 by Ed_Leafe (11,720 points)   1 3 4
0 votes

On 11/02/2017 04:14 PM, Ed Leafe wrote:
On Nov 2, 2017, at 4:17 AM, Dmitry Tantsur dtantsur@redhat.com wrote:

The recording of the call is https://bluejeans.com/s/K3wZZ

Ugh: "To watch the video, you need Adobe Flash Player 10.1 or higher"

I know :( There is a way to download them somewhere there. Look for a small
download sign appearing when you hover the chapter image below.

-- Ed Leafe


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Nov 2, 2017 by Dmitry_Tantsur (18,080 points)   2 3 4
...