Tuesday, 2023-03-28

prometheanfireTheJulia: got part of the way there at least00:58
prometheanfirefetched the image but tried to update the neutron port's dhcp opts and failed because a port is not being created (server is being started with `openstack baremetal deploy foo`00:58
prometheanfirenot sure if there's a way to start a ramdisk deploy via nova (which would be preferred), since it seems to require an image I think00:59
TheJuliaprometheanfire: you have to create the port before deploying with ironic01:00
TheJuliaand then attach with openstack baremetal node vif attach <neutron_port_uuid> <node-id?01:00
TheJulia>01:00
prometheanfireok, I'll look into that, I tried to create a baremetal port with a mac, probably needs more metadata01:00
TheJuliaerr >01:00
TheJuliathere is the baremetal port which ironic holds data about the port in, and then there is the neutron port for dhcp01:00
TheJuliafor provisioning we will create ports, but for end nodes booting we need a vif01:01
TheJuliaThere is a weird nuance there01:01
prometheanfireya, I have the baremetal port created but no vif assocaited with it01:01
TheJulia... we should likely just support a "assign from this" model01:01
prometheanfireso I'll create a neutron port manually and assign that as a vif01:01
TheJulia++01:02
TheJuliayup01:02
prometheanfirejust the mac and baremetal type?01:02
TheJuliayou just need to create a port on the network you want01:02
TheJuliaironic will fix the mac01:02
TheJuliaand the type01:02
prometheanfireah, neat01:02
prometheanfirethen assign from this would be nice :D01:03
TheJuliahttps://i.ytimg.com/vi/LaiN63o_BxA/maxresdefault.jpg01:03
TheJulia... which reminds me... the wife asked about getting a new board and display for one of the printers... *gulp*01:04
prometheanfirepower on, progress :D01:05
TheJulia\o/01:05
prometheanfiream I correct in thinking that ramdisk deploys can't be done via `openstack server create`?01:06
TheJuliaso the idea was that it should work01:06
TheJuliawell, within reason, but we've never tried it01:06
TheJulia... and depending on how the image is tagged, nova might just disqualify it anyway01:06
prometheanfireya, I have no idea how to tell nova to tell ironic to use a kernel/ramdisk only, so meh for now01:07
TheJuliahttps://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L17201:08
prometheanfireohh, neat01:08
TheJuliait is just a uuid in glance, so if there is not actually a disk image behind it01:08
TheJuliathen ironic won't care01:08
TheJuliaas long as the node is set to ramdisk deploy01:08
TheJuliathe thing is... nova doesn't know about toggling that setting so a node has to be set to ramdisk deploy by default01:08
prometheanfirethat part is ok01:09
TheJuliagenerally ramdisk was developed for the kind of folks that do scientific batch processing01:09
TheJuliaas one of those "oh, that would help, sure!"01:09
TheJuliagood will sort of thigns01:09
TheJulia... which turned out generally useful in other was as well01:09
prometheanfireyep, makes sense, cern would like that at the least01:10
TheJuliaI did it after the scientific sig inquired, it made tons of sense01:11
* TheJulia takes her remaining amount of energy for the day to the kitchen to make pizza01:11
* prometheanfire waits for the baby to wake01:12
prometheanfirehmm, need to boot from disk on reboot (not pxe)01:26
TheJuliaWe don’t reassert so if the ramdisk changes the settings locally it should work01:27
prometheanfireya, I think it tried to boot from disk when it rebooted (went to pxe, but the default was local)01:28
prometheanfireit requires some special grub stuff for whatever image is laid down though I think01:29
TheJuliaif uefi, efibootmgr01:34
TheJuliagrub-install used to launch it... but not anymore01:34
prometheanfirehmm, tried to do the ramdisk install again, not boot from local disk01:35
prometheanfiremaybe because I selected it01:35
prometheanfireand it remembered01:36
TheJuliaso we do send the boot from as a persistant override, but the host can change itself back01:36
TheJuliaat least.. I think01:36
TheJuliait is all a bit fuzzy at the moment, not feeling that great :(01:36
prometheanfireya, the docs seem to say that ironic remembers the last manual selection01:36
prometheanfiredos magic invalid, so it's doing something else, back to google01:47
TheJulia:(01:48
prometheanfirehttps://paste.openstack.org/show/819363/ 02:06
prometheanfirefound that, but for now I'm gonna manually select the boot02:06
prometheanfireI can chainloader to it at least, have to edit grub manually02:14
TheJuliaAhh grub pxe template?02:18
TheJuliaprometheanfire: https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/deploy_utils.py02:21
prometheanfire? https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/deploy_utils.py#L44602:33
TheJuliaYes, sorry the line highlight didn’t copy on my paste02:33
prometheanfirethere is an option to force mbr and efi installs, trying that for harvester, if that fails I'll try that02:34
TheJuliaEnjoy!02:34
prometheanfirethanks much for all the help :D02:34
prometheanfireI could edit the template https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/pxe_grub_config.template02:38
prometheanfirealso notice that the boot_whole_disk assumes mbr, so maybe that's it (who knows :D)02:38
prometheanfirehttps://storyboard.openstack.org/#!/story/2004525 is where it was added02:43
prometheanfirecan confirm the pxe_template works, now seeing if I can make the template boot the whole disk on second boot, I don't think there's a way to override the boot line03:49
prometheanfireit hard codes without ability to override on the fly it looks like https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/deploy_utils.py#L130-L18903:50
prometheanfireguess what I'm really looking for is a standard deploy bits to disk, but is managed by a tool other than IPA :|03:55
prometheanfirelooks like they are working on generating qcow2 images https://github.com/harvester/harvester-installer/pull/45705:17
prometheanfirewould make things much easier :D05:17
arne_wiebalckGood morning, Ironic!06:15
jssfrGood morning, Ironic, arne!06:23
rpittaugood morning ironic! o/07:29
opendevreviewDmitry Tantsur proposed openstack/ironic master: Remove all references to the "cpus" property  https://review.opendev.org/c/openstack/ironic/+/87842109:54
iurygregorymorning Ironic12:02
arozman_metal3Hi Ironic! I have been here quite some time ago :D, I am planning to join the PTG I have also registered already, do I need to do anything other than joining the IRC channel ?13:36
TheJuliao/13:37
TheJuliaarozman_metal3: one moment13:37
TheJuliaarozman_metal3: Checkout https://ptg.opendev.org/ptg.html13:38
TheJuliaif you click on the "room", it will launch a meeting client, generally zoom13:38
arozman_metal3ooo, so it is a zoom meeting, I thought just IRC :D, nice13:39
arozman_metal3thanks13:39
* TheJulia attempts to wake up13:41
* TheJulia does not recommend the current chest cold she has13:44
dtantsurthe one we got here is also pretty nasty :(13:57
TheJuliaNowhere like the times I've gotten covid, but I've never had chest congestion build up without a fever before, and now I think as of last night I finally have a fever13:58
dtantsuroh13:58
dtantsursome bacteria is coursing around here.. together with the general shortage of antibiotics, it provides an unpleasant result13:59
dtantsurI hope you recover soon TheJulia!14:00
TheJuliayay14:00
knikollaHope you feel better soon! 14:01
prometheanfireTheJulia: sorry to hear, hope that will be the extent of it14:27
dtantsurade_lee: I pasted links to IPA certificate generation in the etherpad14:35
ade_leedtantsur, thanks - will look14:39
dtantsurJayF: storyboard migration simplified: maybe just switch to launchpad as a place for *new* bugs? and let old stories be fixed/closed/rot?14:44
TheJulia++14:44
TheJuliaI can go push the button if I still have access in launchpad14:45
dtantsurlet's just do it? then we can at least have working bugs from now on?14:46
prometheanfire++14:46
* prometheanfire has a feature request ready :D14:47
rpittaulaunchpad it is then :)14:47
dtantsurarozman_metal3: not sure how much metal3 benefits from sharding.. it's more to scale listing operations. the only listing operation we have is to filter nodes that are in-progress.14:47
dtantsurit could be much more interesting to support the concept of ownership14:47
JayFmetal3+sharding could be interesting for running multiple metal3 off a single Ironic :D 14:48
dtantsurnot sure?14:48
JayFor for carving out a section of ironic for metal3 only (this is probably better off using node.owner ?) 14:48
JayFit's a tool you could use for those things if you wanted14:48
dtantsuryeah, my thought is that we could benefit from ownership14:48
JayFIDK if you want or need to14:48
arozman_metal3there is strong interest for multy tenancy and many different deployment modes of the Metal3 stack downstream14:48
dtantsurarozman_metal3: shards are not a multi-tenancy instrument14:48
dtantsurowners are14:48
dtantsurwe'd need to support Keystone or find another way14:49
dtantsurI don't know if folks are going to scream at us for suggesting supporting Kubernetes authentication :D14:49
arozman_metal3okay good to know, in any case I will mention this as it might be still usefull for us, also the owners , but nowadays Metal3 is being planned to be used in very special deployment envs and configurations thanks to EDGE use cases14:50
arozman_metal3I have even seen POCs where Ironic manages the machine it is running on so everything and anything will be asked from Ironic, (as I have lerned in the last 3-4 months)14:53
* prometheanfire may drop for the osa room14:53
dtantsurwow14:54
dtantsurarozman_metal3: honestly, we need to have our own PTG and dedicate a lot of time to discuss 1) multi-tenancy, 2) multi-conductor, 3) edge14:54
arozman_metal3yes I already have a doc, we have had a long discussion with ESJ team on the 17th, I wanted to bring it to the Metal3 community but I have got sick last week, I will start the discussion this week14:55
JayFdtantsur: arozman_metal3: if only there was someone planning ways to tie in Ironic to external, non-keystone auth systems <.< >.>14:57
JayFdtantsur: does k8s support oauth2.0? The next session is the glue you seek :D14:58
* dtantsur is researching14:58
dtantsurJayF: it does https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens, dunno if it can serve as a provider14:59
JayFeven if it can't provide, being able to back ironic + k8s with the same backend oauth2.0 server would give good behavior I think15:00
samuelkunkel[m]Not to my knowledge. It can use openid connect for its own layer but it only serves as a client. (afaik)15:00
JayFand even if not, keystonemiddleware that does k8s auth is probably possible? idk15:00
samuelkunkel[m]Not 100% but openid connect is just a subset of oauth2, isnt it?15:01
JayFI'm not sure, I'm not really an expert on auth tech15:03
JayFjust excited for interoperability \o/15:03
samuelkunkel[m]But kubernetes auth should be possible (it uses a JWT Token). Just touched it for some hashivault stuff but we used hashivault approles in the end ;)15:05
dtantsurJWT's should be easy to support15:07
dtantsurstevebaker[m]: I'm looking at my KUBECONFIG (from an openshift cloud), and it seems to use mTLS15:09
dtantsursorry, wanted to tag samuelkunkel[m] 15:09
stevebaker[m]:)15:10
dtantsurstevebaker[m]: I'm impressed you're awake though :)15:10
stevebaker[m]I'm feeling good! considering15:11
samuelkunkel[m]dtantsur: https://developer.hashicorp.com/vault/docs/auth/kubernetes15:12
samuelkunkel[m]I was just scrolling through the hashivault documentation as I found it pretty good describing the kubernetes auth15:12
dtantsursamuelkunkel[m]: it seems to be dependent on the exactly kubernetes distribution and the way it is configured :(15:12
samuelkunkel[m](Like mentioned, about a year ago since I touched it)15:13
samuelkunkel[m]Yes and no. I would assume that BoundServiceAccountTokenVolume should be in anything kubernetes alike15:13
samuelkunkel[m]So, from the back of my head, you create a SeviceAccount, associate it to your cr via crb and this Serviceaccount also has a JWT implicit.15:15
samuelkunkel[m]And this can be used for k8s auth15:15
dtantsurI'm reading https://kubernetes.io/docs/reference/access-authn-authz/authentication/#users-in-kubernetes which is not specific to vault/openshift/etc15:15
samuelkunkel[m]* So, from the back of my head, you create a SeviceAccount, associate it to your ClusterRole (or role) via ClusterRoleBinding (or RoleBinding) and this Serviceaccount also has a JWT implicit.15:16
samuelkunkel[m]So, I agree. This is something which can be used for machine2machine auth. I personally would not use it for client authentication.15:17
samuelkunkel[m](We also did not use it for machine2machine)15:18
dtantsuroh fun, kubernetes (or at least openshift) supports keystone :D15:19
JayFyou can use the keystonemiddleware we're talking about to get k8s support for oauth 2.0 /s/s/s15:21
JayF(I know that's not how it works, but it's a funny joke)15:21
dtantsurJust Rewrite It In Go \o/15:22
JayFs/Go/Rust/15:22
JayF2019 called, it wants its cool new language back dtantsur ;) 15:22
dtantsurLOL15:22
JayFif I know how to write code in it, it's officially uncool15:22
dtantsuryou don't need to convince me to s/Go/Rust/15:22
dtantsurarozman_metal3: what's your typical identity backend in your kubernetes installation?15:23
dtantsurI wonder if we should just support LDAP..15:23
samuelkunkel[m]Wasnt the ldap backend in keystone deprecated as no one maintained it?15:24
* JayF just realized he's never worked on an OpenStack cluster running "vanilla" keystone (or anything even vanilla-adjacent)15:24
arozman_metal3dtantsur: do you mean what identity backend is used downstream ?15:25
dtantsurarozman_metal3: yeah, if there is any single typical backend15:25
dtantsurbecause it seems like kubernetes itself is not really opinionated15:25
arozman_metal3dtantsur: I can ask, I don't know tbh15:25
samuelkunkel[m]JayF: what is a vanilla keystone?15:25
JayFas in, not heavily patched15:26
arozman_metal3dtamtsur: I am not allowed access it directly, I always work via intermediary folks15:26
dtantsurI see15:27
samuelkunkel[m]Ah, hmm. I think we use a pretty standard keystone, just run in container.15:27
knikollavanilla, that sounds like a pretty tasty keystone :)15:27
samuelkunkel[m]I think we dont have a single patch in there 15:29
samuelkunkel[m]https://gitlab.com/yaook/images/keystone/-/blob/devel/Dockerfile-train15:29
samuelkunkel[m]* I think we do not have a single patch in there15:29
samuelkunkel[m]https://gitlab.com/yaook/images/keystone/-/blob/devel/Dockerfile-train15:29
samuelkunkel[m]Currently running yoga release15:29
TheJuliaI pasted a simple IDP (literally named simple-idp) into the etherpad15:30
TheJuliawhich gave us a drop-down on the UI15:30
JayFYeah, other places I worked had custom plugins to plugin to custom backend identity things15:34
* JayF has written some really baller internal-CVE-style bugs against them which will never be seen by anyone else :(15:34
knikollaThe more layers in between you and the root of the identity the more fun it gets. 15:39
JayFBreaking until 1615; then going into DPU orchestration16:05
JayFwe're a little ahead16:05
opendevreviewRiccardo Pittau proposed openstack/ironic master: Exclude all files starting with . from flake8 tests  https://review.opendev.org/c/openstack/ironic/+/87878816:11
* prometheanfire wonders if redfish allows setting a boot override (pxe boot ramdisk normally, but set an override per node to boot from disk, or something else)17:00
dtantsurprometheanfire: cannot you use efibootmgr for that?17:12
dtantsursome people use ironic in this fashion: use ramdisk deploy for one-time boot of an installer, then inside the installer, change the UEFI boot order to local disk17:13
prometheanfirecould do that, if the installer supported it lol17:15
prometheanfirehonestly, them generating a qcow2 image would be best, and seems like it's in progress https://github.com/harvester/harvester-installer/pull/45717:16
prometheanfirecan we set values through redfish for random bios values or is it mostly read only?17:18
dtantsurprometheanfire: it's called BIOS settings in ironic17:18
dtantsurhttps://docs.openstack.org/ironic/latest/admin/bios.html17:19
prometheanfirelol17:19
*** sfinucan is now known as stephenfin17:20
rpittaugood night! o/17:32

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!