*** sdake has quit IRC | 00:01 | |
*** vkmc has quit IRC | 00:02 | |
*** rcernin_ has quit IRC | 00:02 | |
*** sdake has joined #openstack-ironic | 00:06 | |
*** sdake has quit IRC | 00:06 | |
*** sdake has joined #openstack-ironic | 00:06 | |
*** jhesketh has quit IRC | 00:06 | |
*** vkmc has joined #openstack-ironic | 00:06 | |
*** lbragstad has quit IRC | 00:06 | |
*** lucasagomes has quit IRC | 00:06 | |
*** rnoriega has quit IRC | 00:06 | |
*** lhinds has quit IRC | 00:06 | |
*** amotoki has quit IRC | 00:06 | |
*** gmann has quit IRC | 00:06 | |
*** jovial[m] has quit IRC | 00:10 | |
*** retr0h has quit IRC | 00:11 | |
*** csmart has quit IRC | 00:11 | |
*** mkrai has quit IRC | 00:11 | |
*** jhesketh has joined #openstack-ironic | 00:15 | |
*** rcernin_ has joined #openstack-ironic | 00:15 | |
*** lbragstad has joined #openstack-ironic | 00:15 | |
*** lucasagomes has joined #openstack-ironic | 00:15 | |
*** rnoriega has joined #openstack-ironic | 00:15 | |
*** lhinds has joined #openstack-ironic | 00:15 | |
*** amotoki has joined #openstack-ironic | 00:15 | |
*** gmann has joined #openstack-ironic | 00:15 | |
*** threestrands has joined #openstack-ironic | 00:23 | |
*** threestrands has quit IRC | 00:23 | |
*** threestrands has joined #openstack-ironic | 00:23 | |
*** livelace2 has quit IRC | 00:28 | |
*** livelace2 has joined #openstack-ironic | 00:28 | |
*** rcernin_ has quit IRC | 00:43 | |
*** rcernin has joined #openstack-ironic | 00:44 | |
*** phuongnh has joined #openstack-ironic | 00:45 | |
*** jovial[m] has joined #openstack-ironic | 00:46 | |
*** exodusftw has quit IRC | 00:46 | |
*** exodusftw has joined #openstack-ironic | 00:52 | |
*** r-daneel has quit IRC | 00:57 | |
*** r-daneel has joined #openstack-ironic | 01:00 | |
*** harlowja has quit IRC | 01:09 | |
*** liuzz has joined #openstack-ironic | 01:14 | |
*** ijw has quit IRC | 01:18 | |
*** r-daneel has quit IRC | 01:18 | |
*** ijw has joined #openstack-ironic | 01:20 | |
*** phuongnh has quit IRC | 01:30 | |
*** phuongnh has joined #openstack-ironic | 01:31 | |
*** gcb has quit IRC | 01:34 | |
*** phuongnh has quit IRC | 01:39 | |
*** phuongnh has joined #openstack-ironic | 01:40 | |
*** tiendc has joined #openstack-ironic | 02:02 | |
*** spartacloud has joined #openstack-ironic | 02:11 | |
*** spartacloud has quit IRC | 02:12 | |
*** retr0h[m] has joined #openstack-ironic | 02:12 | |
*** csmart[m] has joined #openstack-ironic | 02:12 | |
*** madhuri[m] has joined #openstack-ironic | 02:12 | |
*** spartacloud has joined #openstack-ironic | 02:23 | |
*** spartacloud has quit IRC | 02:27 | |
*** spartacloud has joined #openstack-ironic | 02:39 | |
*** links has joined #openstack-ironic | 02:40 | |
*** spartacloud has quit IRC | 02:43 | |
*** lifeless has quit IRC | 02:51 | |
*** threestrands has quit IRC | 02:54 | |
*** lifeless has joined #openstack-ironic | 02:58 | |
*** rh-jelabarre has quit IRC | 03:03 | |
*** rh-jelabarre has joined #openstack-ironic | 03:06 | |
*** ijw has quit IRC | 03:09 | |
*** threestrands has joined #openstack-ironic | 03:10 | |
*** threestrands has quit IRC | 03:10 | |
*** threestrands has joined #openstack-ironic | 03:10 | |
openstackgerrit | Ha Manh Dong proposed openstack/sushy-tools master: Switch to use stestr for unit test https://review.openstack.org/583832 | 03:24 |
---|---|---|
*** trungnv has quit IRC | 04:14 | |
*** ijw has joined #openstack-ironic | 04:24 | |
*** ijw has quit IRC | 04:28 | |
*** harlowja has joined #openstack-ironic | 04:30 | |
*** jaganathan has joined #openstack-ironic | 04:37 | |
*** threestrands has quit IRC | 04:40 | |
*** jiapei has joined #openstack-ironic | 04:44 | |
*** mbeierl has quit IRC | 04:49 | |
*** harlowja has quit IRC | 04:52 | |
*** mbeierl has joined #openstack-ironic | 04:52 | |
*** pcaruana has joined #openstack-ironic | 05:09 | |
*** spartacloud has joined #openstack-ironic | 05:28 | |
*** spartacloud has quit IRC | 05:33 | |
*** dmellado has quit IRC | 05:48 | |
openstackgerrit | Kaifeng Wang proposed openstack/ironic-inspector master: Supports database as introspection data storage: processing https://review.openstack.org/514552 | 05:52 |
*** mjura has joined #openstack-ironic | 06:02 | |
*** skazi has joined #openstack-ironic | 06:15 | |
*** AlexeyAbashkin has joined #openstack-ironic | 06:15 | |
*** ijw has joined #openstack-ironic | 06:25 | |
*** hjensas has joined #openstack-ironic | 06:25 | |
*** hjensas has quit IRC | 06:25 | |
*** hjensas has joined #openstack-ironic | 06:25 | |
*** dmellado has joined #openstack-ironic | 06:29 | |
*** ijw has quit IRC | 06:30 | |
*** tiendc_ has joined #openstack-ironic | 06:30 | |
*** tiendc has quit IRC | 06:33 | |
*** AlexeyAbashkin has quit IRC | 06:41 | |
*** zhangfei has joined #openstack-ironic | 06:48 | |
*** hoonetorg has quit IRC | 06:54 | |
*** AlexeyAbashkin has joined #openstack-ironic | 06:58 | |
*** Alexey_Abashkin has joined #openstack-ironic | 07:01 | |
*** tesseract has joined #openstack-ironic | 07:02 | |
*** AlexeyAbashkin has quit IRC | 07:02 | |
*** Alexey_Abashkin is now known as AlexeyAbashkin | 07:02 | |
*** rcernin has quit IRC | 07:02 | |
*** jtomasek has joined #openstack-ironic | 07:04 | |
*** tiendc has joined #openstack-ironic | 07:05 | |
*** tiendc_ has quit IRC | 07:07 | |
*** hoonetorg has joined #openstack-ironic | 07:11 | |
*** e0ne has joined #openstack-ironic | 07:13 | |
*** hjensas has quit IRC | 07:22 | |
*** jiapei has quit IRC | 07:24 | |
*** hamzy_ has joined #openstack-ironic | 07:31 | |
*** hamzy has quit IRC | 07:31 | |
*** vkmc has quit IRC | 07:32 | |
*** tiendc has quit IRC | 07:32 | |
*** gkadam has joined #openstack-ironic | 07:37 | |
openstackgerrit | Kaifeng Wang proposed openstack/python-ironicclient master: Query node by instance uuid https://review.openstack.org/584263 | 07:39 |
*** spartacloud has joined #openstack-ironic | 07:44 | |
*** spartacloud has quit IRC | 07:49 | |
*** hamzy_ has quit IRC | 07:50 | |
*** alexpilotti has quit IRC | 07:54 | |
*** pmannidi has quit IRC | 07:56 | |
*** spartacloud has joined #openstack-ironic | 07:58 | |
*** jrcloud has joined #openstack-ironic | 08:02 | |
*** spartacloud has quit IRC | 08:03 | |
*** tssurya has joined #openstack-ironic | 08:05 | |
*** dsneddon has quit IRC | 08:06 | |
*** jrcloud has quit IRC | 08:06 | |
*** dsneddon has joined #openstack-ironic | 08:08 | |
*** hkominos_ has joined #openstack-ironic | 08:09 | |
*** ajya has joined #openstack-ironic | 08:10 | |
*** mgoddard has joined #openstack-ironic | 08:12 | |
*** dougsz has joined #openstack-ironic | 08:13 | |
*** zhangfei has quit IRC | 08:14 | |
*** athomas has joined #openstack-ironic | 08:17 | |
openstackgerrit | Kaifeng Wang proposed openstack/ironic-inspector master: [TEST] unit test failure due to oslo.policy upgrade https://review.openstack.org/584273 | 08:17 |
openstackgerrit | Kaifeng Wang proposed openstack/ironic-inspector master: Stop introspection if set boot device failed https://review.openstack.org/583519 | 08:20 |
*** zhangfei has joined #openstack-ironic | 08:26 | |
*** hjensas has joined #openstack-ironic | 08:50 | |
*** hjensas has quit IRC | 08:50 | |
*** hjensas has joined #openstack-ironic | 08:50 | |
*** liuzz has quit IRC | 09:08 | |
*** liuzz has joined #openstack-ironic | 09:11 | |
*** liuzz has quit IRC | 09:13 | |
*** liuzz has joined #openstack-ironic | 09:13 | |
*** mgoddard has quit IRC | 09:17 | |
*** alexpilotti has joined #openstack-ironic | 09:18 | |
*** skazi has quit IRC | 09:24 | |
*** links has quit IRC | 09:29 | |
*** hamzy_ has joined #openstack-ironic | 09:31 | |
openstackgerrit | Kaifeng Wang proposed openstack/ironic-inspector master: Supports database as introspection data storage: processing https://review.openstack.org/514552 | 09:35 |
*** csmart[m] has quit IRC | 09:42 | |
*** madhuri[m] has quit IRC | 09:42 | |
*** retr0h[m] has quit IRC | 09:43 | |
*** jovial[m] has quit IRC | 09:43 | |
*** prasen has joined #openstack-ironic | 09:48 | |
*** vkmc has joined #openstack-ironic | 09:49 | |
*** jovial[m] has joined #openstack-ironic | 09:55 | |
*** liuzz has quit IRC | 09:56 | |
*** hjensas has quit IRC | 10:00 | |
*** mgoddard has joined #openstack-ironic | 10:04 | |
*** AlexeyAbashkin has quit IRC | 10:08 | |
*** skazi has joined #openstack-ironic | 10:09 | |
*** AlexeyAbashkin has joined #openstack-ironic | 10:11 | |
*** spartacloud has joined #openstack-ironic | 10:17 | |
*** AlexeyAbashkin has quit IRC | 10:18 | |
*** spartacloud has quit IRC | 10:22 | |
*** AlexeyAbashkin has joined #openstack-ironic | 10:31 | |
*** dtantsur|afk is now known as dtantsur | 10:31 | |
dtantsur | morning ironic | 10:31 |
*** phuongnh has quit IRC | 10:32 | |
etingof | good Friday morning everyone o/ | 10:35 |
*** retr0h[m] has joined #openstack-ironic | 11:01 | |
*** madhuri[m] has joined #openstack-ironic | 11:01 | |
*** csmart[m] has joined #openstack-ironic | 11:01 | |
*** hjensas has joined #openstack-ironic | 11:04 | |
*** hjensas has quit IRC | 11:04 | |
*** hjensas has joined #openstack-ironic | 11:04 | |
*** ijw has joined #openstack-ironic | 11:24 | |
*** ijw has quit IRC | 11:29 | |
*** jaypipes is now known as leakypipes | 11:32 | |
*** links has joined #openstack-ironic | 11:32 | |
jroll | morning, happy friday \o/ | 11:37 |
dtantsur | hey jroll | 11:38 |
jroll | \o dtantsur | 11:38 |
dtantsur | responded on the conductor group patch | 11:41 |
dtantsur | now I cannot get why you need two hash rings.. | 11:42 |
jroll | right, I will go back over it today | 11:44 |
jroll | we use undelimited so that rolling upgrade is smooth | 11:44 |
jroll | and we agreed that storing the delimited now will allow us a smooth switch to delimited in stein | 11:44 |
jroll | but I'm not sure that is true | 11:45 |
dtantsur | yeah, I don't see how it's true, given that, as you rightfully notice, a hash ring is local | 11:46 |
*** kiseok7 has quit IRC | 11:48 | |
dtantsur | srsly, we never implemented this https://review.openstack.org/#/c/584263/ ? O__o | 11:53 |
patchbot | patch 584263 - python-ironicclient - Query node by instance uuid | 11:53 |
TheJulia | Good morning everyone | 11:53 |
TheJulia | dtantsur: I believe the nova virt driver code does it via get get nodes with an argument | 11:54 |
TheJulia | It also has the weird abstraction that it uses | 11:54 |
dtantsur | morning TheJulia | 11:54 |
TheJulia | dtantsur: so I got an aarch64 vm running last night (was super painful) | 11:55 |
dtantsur | TheJulia: \o/ awesome | 11:56 |
dtantsur | I suggest we add this ironicclient patch to the upcoming release | 11:56 |
TheJulia | so I think it wouldn't be too difficult to build testing around both uefi paths ... to assist in ripping out elilo support. | 11:56 |
TheJulia | dtantsur: agreed | 11:56 |
jroll | FYI we already have SDK code for it https://github.com/openstack/python-ironicclient/blob/master/ironicclient/v1/node.py#L326 | 11:56 |
jroll | not sure if we want to use that instead of adding an argument to list() | 11:57 |
dtantsur | jroll: mmm, right. but we still need to expose it in CLI | 11:57 |
jroll | https://github.com/openstack/python-ironicclient/blob/master/ironicclient/osc/v1/baremetal_node.py#L1224 | 11:58 |
jroll | looks like it's there | 11:58 |
jroll | openstack baremetal node show --instance foo | 11:58 |
dtantsur | oh, and we do, yeah | 11:58 |
jroll | :) | 11:58 |
* dtantsur changes his vote | 11:58 | |
jroll | I had the same O__o reaction, I knew it couldn't be true :P | 11:58 |
dtantsur | good catch jroll :) | 11:59 |
jroll | :) | 12:00 |
*** zhangfei has quit IRC | 12:00 | |
*** jaganathan has quit IRC | 12:01 | |
TheJulia | dtantsur: w/r/t the new option I proposed to add for using macs for uefi. It has a +2 right now, we could land it and deprecate/kill it next cycle. I don't think there is going to be any way to get all of the uefi stuff sorted before we cut this cycle's release, and switching to mac default via removing elilo is likely moderately logically and then write out an ip address for compatibility if neutron is turned on | 12:01 |
*** arnewiebalck__ has joined #openstack-ironic | 12:02 | |
dtantsur | TheJulia: is it possible to detect if we're using elilo or not? | 12:03 |
TheJulia | dtantsur: w/r/t to the two hash rings are needed so api nodes can understand how to handle the transition to delimited once conductors start getting upgraded. Then again, we might be seriously overthinking it, but changing the structure with adding the delimiter is breaking to api->conductor communication ability by for topic selection. | 12:04 |
*** trown|outtypewww is now known as trown | 12:04 | |
dtantsur | TheJulia: something like https://github.com/openstack/ironic/blob/master/ironic/common/pxe_utils.py#L239 | 12:04 |
TheJulia | dtantsur: I don't think so, but I've also not even tried to find elilo packages for a long time | 12:04 |
dtantsur | which is one place we're already doing that | 12:05 |
TheJulia | https://www.rpmfind.net/linux/rpm2html/search.php?query=elilo lol | 12:05 |
dtantsur | TheJulia: wrt the hash rings, I don't think this is actually true | 12:05 |
dtantsur | as Jim reminded me, any API instance is building its own hash ring, with or without delimiters | 12:05 |
TheJulia | I guess it all depends on how smooth we want to make the transition. | 12:05 |
dtantsur | so I don't quite understand why we care | 12:05 |
TheJulia | indeed, and addresses those messages, so if we want things to match up correctly it seems necessary. If we don't care as much *shrugs* | 12:06 |
dtantsur | I seem to be missing some bit part of it. if any service has its own hash ring copy, why do we care about delimiters? | 12:06 |
jroll | there's two goals: keep them in sync as much as possible, and reduce take_over as much as possible | 12:06 |
jroll | using the undelimited for this upgrades means the hash ring does not change during the upgrade, which helps with both of these | 12:06 |
jroll | (because no nodes or conductors have a group yet) | 12:07 |
TheJulia | ahh, yeah, rloo and us also talked about take_over | 12:07 |
*** MattMan has quit IRC | 12:07 | |
*** prasen has quit IRC | 12:07 | |
*** MattMan has joined #openstack-ironic | 12:07 | |
dtantsur | oh. anyway, the current code has to be fixed, since one of the rings is never used. or is it? | 12:08 |
jroll | now for the upgrade to stein, we do want to switch to using a delimited hash ring, because it reduces the opportunity for conflicts (example group "ip" and hw type "mi" conflicts with group "" and hw type "ipmi") | 12:08 |
jroll | we somehow agreed that storing both now helps us do that upgrade more smoothly. but now I really can't remember how. | 12:09 |
dtantsur | yeah, that's my point. one of the rings is generated (in-memory) and immediately discarded | 12:10 |
dtantsur | only the undelimited ring seems used | 12:10 |
dtantsur | so it does not seem to me that we can avoid take_overs on upgrade to Stein when the rings start being delimited.. | 12:10 |
jroll | right | 12:10 |
jroll | in which case we should just do it now | 12:10 |
dtantsur | or even: maybe take_over's are not a problem at all | 12:11 |
dtantsur | again, each conductor detects if it manages a host | 12:11 |
dtantsur | mmm, interesting, I wonder if two conductors (old and new) can start fighting for a host | 12:12 |
* dtantsur starts getting headache again | 12:12 | |
jroll | right, that's the concern, the time during rolling restarts to unpin it | 12:12 |
dtantsur | mmm, okay, no, at each point in time both undelimited and delimited hash rings are identical | 12:13 |
dtantsur | because the database does not know about delimiters... | 12:14 |
jroll | oh, the hash ring itself only deals with conductor names and node uuids | 12:15 |
dtantsur | wait the database does know about delimiters | 12:15 |
dtantsur | jroll: isn't https://review.openstack.org/#/c/579684/13/ironic/db/sqlalchemy/api.py@858 breaking the driver API? | 12:15 |
patchbot | patch 579684 - ironic - Use conductor group for hash ring calculations | 12:15 |
jroll | oof | 12:16 |
dtantsur | jroll: we're using it in https://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/driver.py#L385 | 12:16 |
jroll | yeah, I realized that when you said it | 12:16 |
dtantsur | wtf don't we have tempest tests for these endpoints? :) | 12:16 |
jroll | not ones where the conductors are configured to use a group :) | 12:17 |
dtantsur | riiiiight | 12:17 |
jroll | sigh, lots of problems here | 12:17 |
dtantsur | well, solvable problems | 12:18 |
jroll | yeah | 12:18 |
jroll | I'd rather be doing something else though :D | 12:18 |
dtantsur | heh | 12:18 |
dtantsur | imagine my mood when fixing unit tests for the driver composition reform? :D | 12:19 |
dtantsur | you're at least developing new cool features | 12:19 |
jroll | yeah, I bet | 12:19 |
*** early has quit IRC | 12:19 | |
dtantsur | a small request: let's concentrate all the delimiter logic in one place. I nearly missed this dbapi change. | 12:20 |
openstackgerrit | vinay kumar muddu proposed openstack/ironic master: iLO BIOS interface implementation https://review.openstack.org/576803 | 12:20 |
jroll | yeah, I agree with that | 12:20 |
dtantsur | "Temperatures will continue to rise and by the end of the week could reach upwards of 35 to 37 degrees." dear weather, why the hell you did not do it when I was on vacation? >_< | 12:21 |
jroll | ouch | 12:21 |
dtantsur | if next week I have bad internet, you know that I work-from-sea :D | 12:22 |
jroll | sounds great | 12:22 |
*** early has joined #openstack-ironic | 12:23 | |
*** dtantsur is now known as dtantsur|brb | 12:23 | |
TheJulia | work-from-sea sounds... interesting | 12:23 |
TheJulia | wheeeee.... all my weekend plans just imploded | 12:25 |
jroll | :/ | 12:26 |
jroll | are they suddenly the same as my weekend plans? | 12:26 |
jroll | "do more work on the house" | 12:26 |
TheJulia | well, I guess the one on wheels needs re-packing | 12:26 |
*** skazi has quit IRC | 12:26 | |
jroll | ah | 12:27 |
*** AlexeyAbashkin has quit IRC | 12:31 | |
* TheJulia finishes first coffee | 12:31 | |
*** AlexeyAbashkin has joined #openstack-ironic | 12:35 | |
*** dprince has joined #openstack-ironic | 12:41 | |
*** d0ugal has quit IRC | 12:45 | |
*** d0ugal has joined #openstack-ironic | 12:45 | |
*** d0ugal has quit IRC | 12:45 | |
*** d0ugal has joined #openstack-ironic | 12:45 | |
*** edleafe is now known as figleaf | 12:46 | |
hkominos_ | Hi all. Can I get some quick input on this error ? https://paste.fedoraproject.org/paste/VktDmaS5FssPdhsPrU30Bg/ I think it implies that a package is missing on my images but I would like to verify | 12:47 |
*** arnewiebalck__ has quit IRC | 12:53 | |
*** rpioso|afk is now known as rpioso | 12:55 | |
rpioso | Good morning | 12:55 |
*** tssurya has quit IRC | 12:55 | |
*** AlexeyAbashkin has quit IRC | 12:55 | |
*** AlexeyAbashkin has joined #openstack-ironic | 12:56 | |
*** d0ugal has quit IRC | 12:57 | |
TheJulia | good morning rpioso | 13:00 |
openstackgerrit | Kaifeng Wang proposed openstack/ironic-inspector master: Fix gate failure due to recent oslo.policy release https://review.openstack.org/584273 | 13:00 |
rpioso | TheJulia: o/ | 13:00 |
openstackgerrit | Julia Kreger proposed openstack/bifrost master: WIP Arm64 https://review.openstack.org/584363 | 13:01 |
*** ijw has joined #openstack-ironic | 13:01 | |
TheJulia | hkominos_: that... seems... very... broken | 13:02 |
TheJulia | heh you'r using an aarch64 machine, how did you build the image? Asking because I'm going to be approaching the same problem very soon | 13:03 |
*** arnewiebalck__ has joined #openstack-ironic | 13:04 | |
TheJulia | hkominos_: do you know if the arm64 grub modules were installed? | 13:04 |
*** jcoufal has joined #openstack-ironic | 13:05 | |
*** ijw has quit IRC | 13:06 | |
*** d0ugal has joined #openstack-ironic | 13:09 | |
* TheJulia wonders if people are selling arm gear with BMCs at a reasoable price these days | 13:13 | |
hkominos_ | TheJulia: Hi. So the good news is that It kinda works on aarch64. At least my virtual deploy worked fine and I use DIB tobuild the images | 13:16 |
hkominos_ | TheJulia: When I use whole-images the system boots fine. | 13:16 |
hkominos_ | TheJulia: The point is that when I use DIB for a whole disk image I define myself hwo the disk will look like. But If i understand correctly when you provide ironic with rootfs,kernel, and ramdisk, it is ironic which will create the partitions etc.. | 13:18 |
*** rloo has joined #openstack-ironic | 13:18 | |
hkominos_ | TheJulia: So regarding your question: do you know if the arm64 grub modules were installed? -->the question is where ? should grub2 existin in the ironic python agent ramdisk or exist within my rootfs ? | 13:19 |
TheJulia | hkominos_: That it should as long as your not explicitly set to netboot | 13:19 |
TheJulia | hkominos_: both I believe | 13:19 |
hkominos_ | TheJulia: Ok I will try with some new images then. The issue is that grub2 package has a weird name on aarch64. and that is probably why he cannto find it | 13:20 |
hkominos_ | TheJulia: Should I assume that if the package is already present in the images, ironic will not complain??? | 13:20 |
TheJulia | hkominos_: it looks like part of it is present, the other part seems not | 13:21 |
TheJulia | it is when ipa is calling grub-install... | 13:21 |
TheJulia | Hmm.. here is one, we're lacking aarch64 templates... | 13:21 |
TheJulia | for network booting that is | 13:21 |
*** arnewiebalck__ has quit IRC | 13:22 | |
hkominos_ | TheJulia: Ok. I will install grub2 manually inside the images and pray: thx for your help | 13:23 |
TheJulia | hkominos_: just make sure you get the grub2 arm efi modules as well | 13:23 |
hkominos_ | TheJulia. I do. I have added this "grub-efi-aarch64": "grub2-efi-aa64 grub2-efi-aa64-modules", in https://github.com/openstack/diskimage-builder/blob/master/diskimage_builder/elements/bootloader/pkg-map | 13:26 |
hkominos_ | TheJulia: there is this strange naming convention between aarch64 ,armv8, arm64 which I dont really understand. | 13:27 |
TheJulia | :( | 13:27 |
hkominos_ | TheJulia: I forgot to ask. Does the error above mean that grub2 is missing from the target image or from the Ironic-python image? | 13:31 |
TheJulia | I think the modules from the ipa image, but you'll need it in the deployed image if anything needs to be changed after deployment | 13:31 |
MrBerns | Hey guys, what is the proper way to delete instances on bifrost? The openstack API commands arent avaialble as nova isnt really deployed | 13:36 |
jroll | rloo: morning, sorry to bother you about it again. we can't figure out how keeping two hash rings helps with the upgrade to stein: http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-ironic.2018-07-20.log.html#t2018-07-20T12:04:16 | 13:37 |
rloo | good morning jroll. | 13:37 |
*** rloo is now known as waterloo | 13:38 | |
jroll | MrBerns: openstack baremetal node undeploy $node_uuid | 13:38 |
jroll | though | 13:38 |
jroll | MrBerns: there's extra things to take care of, I wonder if bifrost has an all in one command, let me look | 13:38 |
waterloo | looking, i thought we had discussed later that we didn't need two rings... give me a few min | 13:38 |
MrBerns | Thanks jroll | 13:38 |
jroll | waterloo: thanks, wanted to check before I rewrite it all again :/ | 13:39 |
*** zhangfei has joined #openstack-ironic | 13:39 | |
TheJulia | jroll: ironic deletes instance_info and all | 13:40 |
*** TheJulia is now known as needssleep | 13:40 | |
* needssleep truly needs more sleep today | 13:40 | |
jroll | needssleep: so what I said works? | 13:40 |
needssleep | yes, it should just fine | 13:40 |
jroll | cool, MrBerns ^^ | 13:40 |
MrBerns | Thanks guys Ill give that command a go | 13:41 |
waterloo | jroll: after reading that exchange, the short answer is 'i don't know for sure'. When do we need to get that feature in by? | 13:43 |
MrBerns | jroll, needssleep Thanks, that seems to have done the trick | 13:43 |
waterloo | feature freeze is july 23. next week. | 13:43 |
jroll | waterloo: well, hopefully next week to make feature freeze and client freeze | 13:43 |
jroll | ya | 13:43 |
jroll | or we could revert the half that landed I guess | 13:43 |
needssleep | Or we could roll forward | 13:44 |
waterloo | hmm. i feel like i would need to spend 2-3 hours to be sure | 13:44 |
jroll | yeah I guess | 13:44 |
*** jcoufal has quit IRC | 13:44 | |
needssleep | we've ripped stuff back out and it quite literally shot us in the foot for a long time | 13:44 |
* needssleep things tags patches | 13:44 | |
jroll | mark the config with something about no effect | 13:44 |
waterloo | has it all landed? | 13:44 |
jroll | the db/objects code landed, and the conductor config | 13:44 |
needssleep | jroll: that is reasonable | 13:44 |
needssleep | but I'd still like to get minimally viable in if at all possible since we could continue to roll forward | 13:45 |
*** jcoufal has joined #openstack-ironic | 13:45 | |
jroll | honestly, we can probably just go straight to the delimited hash ring and nobody would notice any effects that has on rolling upgrades | 13:45 |
jroll | (with an rpc version bump) | 13:45 |
waterloo | just thinking. i wasn't planning on looking at that today. can i look at it on Monday, or i guess you'd prefer i think about it today? | 13:46 |
needssleep | jroll: the only way I think it could be noticed... is if they had something already weirdly partitioned in an unsupported way | 13:46 |
jroll | waterloo: today would be better but it's your call | 13:46 |
waterloo | jroll: i'll look at it later today, ok. but as-is, i don't think we need to roll back. only the config is exposed. | 13:46 |
*** mbeierl has quit IRC | 13:46 | |
jroll | needssleep: idk how one would do that | 13:46 |
*** mjura has quit IRC | 13:46 | |
*** mjura has joined #openstack-ironic | 13:47 | |
needssleep | jroll: hacked driver names | 13:47 |
jroll | eh, that should be fine | 13:48 |
needssleep | not if pools of conductors exist without the other drivers | 13:49 |
needssleep | hacked drivers that is | 13:49 |
needssleep | but we're deep in theoretical land | 13:50 |
needssleep | lets focus on the 90% practical | 13:50 |
*** dprince has quit IRC | 13:54 | |
*** dprince has joined #openstack-ironic | 13:55 | |
*** ijw has joined #openstack-ironic | 13:59 | |
*** mjturek has joined #openstack-ironic | 14:03 | |
*** dansmith is now known as superdan | 14:04 | |
*** ijw has quit IRC | 14:04 | |
*** tssurya has joined #openstack-ironic | 14:04 | |
*** arnewiebalck__ has joined #openstack-ironic | 14:05 | |
*** markguz has joined #openstack-ironic | 14:05 | |
*** markguz_ has joined #openstack-ironic | 14:09 | |
*** mjturek has quit IRC | 14:10 | |
*** markguz has quit IRC | 14:12 | |
*** spartacloud has joined #openstack-ironic | 14:13 | |
*** jcoufal has quit IRC | 14:14 | |
*** mjturek has joined #openstack-ironic | 14:14 | |
*** jcoufal has joined #openstack-ironic | 14:14 | |
*** dtantsur|brb is now known as dtantsur | 14:15 | |
*** spartacloud has quit IRC | 14:18 | |
dtantsur | ++ I think we can land it | 14:18 |
arnewiebalck__ | Hi Ironic, is there any advise on how to deploy the inspector wrt to the api or the conductor? What we did is to create “all-in-one” controller nodes (api, conductor, inspector all in one VM) and have multiple of these. The idea was to easily scale horizontally and add or remove these building blocks when needed. Seems to work fine so far (~1200 nodes). However: we just noticed that the inspector does a | 14:19 |
arnewiebalck__ | regular clean up where it gets all nodes … and this is now done on all controllers :-) No big deal, just unnecessary … and it may cause some issue as we’ll need to scale this up by a factor of ~10, so I was wondering if there were some best practices, for instance if the inspector should be kept separate. | 14:19 |
NobodyCam | Good Morning Ironic'ers ... | 14:20 |
NobodyCam | and .. | 14:20 |
NobodyCam | ofc .. | 14:21 |
NobodyCam | TGIF | 14:21 |
*** ijw has joined #openstack-ironic | 14:24 | |
dtantsur | hey hey NobodyCam, TGIF indeed | 14:24 |
NobodyCam | :) oh ya :) I'm ready for the weekend too.. New pupper has been keeping everyone awake all night :p | 14:25 |
NobodyCam | How about you any plans? | 14:25 |
dtantsur | arnewiebalck__: are you using inspector in HA mode? if not, you need only one instance. otherwise there is not much point to get beyond 3 instances for HA. | 14:26 |
dtantsur | NobodyCam: heh, puppers are like that :) dunno about plans, would go to the sea (it's going to be HOT here), but the budget is a bit tight. | 14:26 |
dtantsur | you? | 14:26 |
arnewiebalck__ | dtantsur: yes, and I agree | 14:27 |
arnewiebalck__ | dtantsur: this is mostly b/c it’s simple and allows to have symmetric controllers | 14:27 |
arnewiebalck__ | dtantsur: we have 5 controllers atm | 14:27 |
NobodyCam | sleep! and some work around the house. :P not to hot here only about 95 F here this weekend | 14:28 |
arnewiebalck__ | dtantsur: 5 to handle power sync :) | 14:28 |
*** jtomasek has quit IRC | 14:28 | |
*** ijw has quit IRC | 14:29 | |
dtantsur | NobodyCam: 35C? I consider it hot :) | 14:29 |
arnewiebalck__ | dtantsur: I noticed constant load and had a look where this was coming from | 14:29 |
arnewiebalck__ | dtantsur: (inspector clean_up) | 14:29 |
dtantsur | arnewiebalck__: yeah, power sync is.. problematic | 14:29 |
dtantsur | hmm, clean_up? it can be tuned if it causes problems | 14:29 |
arnewiebalck__ | dtantsur: yes, we can increase the interval | 14:29 |
arnewiebalck__ | it finishes in 53 secs | 14:30 |
*** tssurya has quit IRC | 14:30 | |
arnewiebalck__ | takes a break of 7 :-D | 14:30 |
arnewiebalck__ | dtantsur: power sync is the root of all evil :) | 14:30 |
arnewiebalck__ | dtantsur: as mentioned, no big deal, just wondering if the whole setup is … suboptimal | 14:31 |
*** efried is now known as fried_rice | 14:32 | |
arnewiebalck__ | dtantsur: the load is coming from policy checking it seems | 14:32 |
arnewiebalck__ | dtantsur: for every node, oslo is checking the policy, after searching for it, so there are quite some stats involved | 14:33 |
*** ajya has quit IRC | 14:33 | |
dtantsur | ouch | 14:34 |
dtantsur | I guess it's inevitable since ironic and ironic-inspector are different services.. | 14:34 |
arnewiebalck__ | yes | 14:34 |
dtantsur | arnewiebalck__: if you don't delete nodes too often, you may lower the rate of inspector clean up | 14:35 |
*** mjturek has quit IRC | 14:35 | |
dtantsur | as to power sync.. needssleep is doing something to allow splitting periodic tasks into their own processes | 14:35 |
arnewiebalck__ | dtantsur: what does the clean up actually clan up? | 14:36 |
needssleep | well, more like be able to run multiple conductors, but yeah | 14:36 |
needssleep | its a massive change though | 14:36 |
dtantsur | arnewiebalck__: I think the primary goal is to remove nodes from internal cache that are removed from ironic | 14:36 |
arnewiebalck__ | dtantsur: ok | 14:36 |
dtantsur | maybe timeouts check are in the same task, I'll double-check | 14:36 |
arnewiebalck__ | needssleep: the issue with power sync was mostly that nova was hammering the iornic api for 1000+ instances | 14:38 |
needssleep | arnewiebalck__: Interesting, I wasn't aware that was presenting as a problem for deployments | 14:38 |
arnewiebalck__ | needssleep: got better when we reduced the green threads pool and added 2 more controllers | 14:39 |
dtantsur | arnewiebalck__: yeah, the same task also checks timeouts. but it should be quicker, it only accesses the DB | 14:39 |
arnewiebalck__ | needssleep: power sync? | 14:39 |
needssleep | arnewiebalck__: well, power sync from the conductors standpoint while it is walking the individual nodes to check the status | 14:39 |
arnewiebalck__ | needssleep: no, I meant nova getting the power state | 14:40 |
needssleep | arnewiebalck__: please tell me you've submitted a talk to the berlin summit on your performance issues and actions taken? | 14:40 |
needssleep | hmmmm | 14:40 |
arnewiebalck__ | needssleep: I have! | 14:40 |
needssleep | arnewiebalck__: AWESOME! | 14:41 |
dtantsur | \o/ | 14:41 |
* dtantsur is ready for voting | 14:41 | |
needssleep | arnewiebalck__: I feel the need to look at the virt driver code.... but I'm kind of on this track of building us some better uefi testing | 14:41 |
needssleep | arm based at that... | 14:41 |
arnewiebalck__ | needssleep: sounds great | 14:42 |
arnewiebalck__ | dtantsur: :) thx! | 14:43 |
needssleep | somebody please remind me if I don't look at that in the next couple of weeks, since we won't be able to really improve that performance until next cycle | 14:44 |
* needssleep wonders if that is what china telecom was also encountering | 14:44 | |
needssleep | err | 14:44 |
needssleep | china mobile | 14:44 |
* dtantsur has this spec next up for review | 14:45 | |
needssleep | dtantsur: conductor? | 14:47 |
*** mjturek has joined #openstack-ironic | 14:48 | |
*** mjturek has quit IRC | 14:50 | |
*** links has quit IRC | 14:52 | |
dtantsur | periodics workers | 14:56 |
needssleep | k | 14:56 |
*** rpittau_ has quit IRC | 14:59 | |
*** e0ne has quit IRC | 14:59 | |
*** r-daneel has joined #openstack-ironic | 14:59 | |
*** ijw has joined #openstack-ironic | 15:00 | |
*** zhangfei has quit IRC | 15:01 | |
*** lbragstad has quit IRC | 15:03 | |
*** r-daneel_ has joined #openstack-ironic | 15:04 | |
*** r-daneel has quit IRC | 15:04 | |
*** r-daneel_ is now known as r-daneel | 15:04 | |
*** ijw has quit IRC | 15:05 | |
*** mjturek has joined #openstack-ironic | 15:05 | |
*** e0ne has joined #openstack-ironic | 15:05 | |
*** e0ne has quit IRC | 15:06 | |
*** e0ne has joined #openstack-ironic | 15:06 | |
*** mjturek has quit IRC | 15:07 | |
*** bnemec is now known as beekneemech | 15:09 | |
*** mjturek has joined #openstack-ironic | 15:10 | |
*** lbragstad has joined #openstack-ironic | 15:12 | |
*** arnewiebalck__ has quit IRC | 15:13 | |
*** mjura has quit IRC | 15:18 | |
*** arnewiebalck_ has quit IRC | 15:19 | |
*** hoonetorg has quit IRC | 15:24 | |
*** hoonetorg has joined #openstack-ironic | 15:25 | |
openstackgerrit | Michael Turek proposed openstack/ironic-python-agent master: Install grub to PReP partition when prep_boot_part_uuid is provided https://review.openstack.org/582962 | 15:51 |
*** trown is now known as trown|lunch | 15:51 | |
*** r-daneel has quit IRC | 15:53 | |
*** r-daneel has joined #openstack-ironic | 15:53 | |
openstackgerrit | Michael Turek proposed openstack/ironic-python-agent master: Install grub to PReP partition when prep_boot_part_uuid is provided https://review.openstack.org/582962 | 16:08 |
*** AlexeyAbashkin has quit IRC | 16:13 | |
*** e0ne has quit IRC | 16:14 | |
openstackgerrit | Debayan Ray proposed openstack/sushy master: Add system storage resource support https://review.openstack.org/561863 | 16:16 |
*** med_ has joined #openstack-ironic | 16:19 | |
*** med_ has quit IRC | 16:19 | |
*** med_ has joined #openstack-ironic | 16:19 | |
*** tesseract has quit IRC | 16:24 | |
*** fried_rice is now known as fried_rolls | 16:24 | |
*** gkadam has quit IRC | 16:29 | |
*** MrBerns has quit IRC | 16:30 | |
*** mgoddard has quit IRC | 16:34 | |
*** tssurya has joined #openstack-ironic | 16:41 | |
openstackgerrit | Merged openstack/ironic-specs master: The direct deploy interface provisioning with HTTP server https://review.openstack.org/504039 | 16:51 |
*** dougsz has quit IRC | 16:55 | |
*** vmud213 has joined #openstack-ironic | 17:10 | |
dtantsur | okay, have a great weekend everyone! | 17:11 |
*** dtantsur is now known as dtantsur|afk | 17:11 | |
needssleep | have a wonderful weekend | 17:11 |
*** trown|lunch is now known as trown | 17:12 | |
*** vmud213 has quit IRC | 17:20 | |
*** spartacloud has joined #openstack-ironic | 17:26 | |
*** jtomasek has joined #openstack-ironic | 17:28 | |
openstackgerrit | Merged openstack/bifrost master: Remove syslinux usage https://review.openstack.org/582655 | 17:30 |
*** racedo has quit IRC | 17:30 | |
*** r-daneel has quit IRC | 17:39 | |
*** spartacloud has quit IRC | 17:40 | |
*** jtomasek has quit IRC | 17:47 | |
*** jtomasek has joined #openstack-ironic | 17:49 | |
*** jtomasek has quit IRC | 17:54 | |
*** med_ has quit IRC | 18:00 | |
*** tssurya has quit IRC | 18:02 | |
*** kzaitsev1pi has joined #openstack-ironic | 18:14 | |
*** tssurya has joined #openstack-ironic | 18:15 | |
*** kzaitsev_pi has quit IRC | 18:17 | |
*** e0ne has joined #openstack-ironic | 18:18 | |
waterloo | jroll: i commented on https://review.openstack.org/#/c/579684/13 | 18:20 |
patchbot | patch 579684 - ironic - Use conductor group for hash ring calculations | 18:20 |
waterloo | jroll: so that's what i meant, but i didn't (don't) know if that solves the problem. it was my thought at the time. I will think about it more now, wrt rolling upgrades. | 18:21 |
jroll | waterloo: thanks | 18:22 |
jroll | I'm not sure which would be the trigger to look it up as delimited or not | 18:22 |
waterloo | jroll: the rpc api version. at least, that is what i was thinking of at the time. | 18:24 |
jroll | waterloo: that still gives us the same issue when rolling out the unpin | 18:24 |
waterloo | jroll: no, or err, i need to convince myself. but with old code OR pin: no delimiter; old code & new code will return same value. | 18:25 |
jroll | waterloo: right, but when conductor1 is unpinned and conductor2 is still pinned, it's going to be different | 18:26 |
jroll | that said, we realized the keys there have nothing to do with the data in the ring itself | 18:26 |
waterloo | oh, that's what i have to do. really understand what the rings/data represent. | 18:27 |
waterloo | we should have documented it :-( | 18:27 |
needssleep | I might be able to at some point, but I'll need tequila and more time in the universe | 18:28 |
jroll | waterloo: this is the code for the actual ring, we just keep some number of them in ironic's data structure | 18:29 |
jroll | so that said, I'm not sure the keys actually matter much | 18:31 |
openstackgerrit | Debayan Ray proposed openstack/sushy master: Add storage and simple_storage attr to system https://review.openstack.org/563051 | 18:31 |
waterloo | jroll: i will have to disappear to think about this (to prove to myself one way or the other). | 18:32 |
jroll | waterloo: fair enough, I'm going to go back to only using the delimited version for now | 18:32 |
waterloo | jroll: ok, I don't think we want two rings. Whether we have one ring with one key per driver, or one ring with two keys per driver (one w/o conductor group) is what I am wondering about. | 18:34 |
jroll | right, I see what you were saying now | 18:34 |
jroll | afaict it's all local so nothing matters :) | 18:34 |
waterloo | jroll: you're right. delimited or not delimited, the value is the same. | 18:36 |
*** r-daneel has joined #openstack-ironic | 18:37 | |
*** fried_rolls is now known as fried_rice | 18:38 | |
*** markguz has joined #openstack-ironic | 18:39 | |
*** markguz_ has quit IRC | 18:43 | |
*** ijw has joined #openstack-ironic | 18:44 | |
*** ijw has quit IRC | 18:44 | |
*** athomas has quit IRC | 18:50 | |
*** tssurya has quit IRC | 18:58 | |
*** jcoufal has quit IRC | 19:00 | |
*** tssurya has joined #openstack-ironic | 19:00 | |
*** e0ne has quit IRC | 19:08 | |
*** arnewiebalck_ has joined #openstack-ironic | 19:10 | |
*** ijw has joined #openstack-ironic | 19:12 | |
*** med_ has joined #openstack-ironic | 19:13 | |
*** med_ has quit IRC | 19:13 | |
*** med_ has joined #openstack-ironic | 19:13 | |
*** tssurya has quit IRC | 19:16 | |
*** tssurya has joined #openstack-ironic | 19:16 | |
jroll | blah, not sure I can leave the grouping stuff out of dbapi | 19:24 |
*** markguz has quit IRC | 19:24 | |
*** markguz has joined #openstack-ironic | 19:25 | |
*** e0ne has joined #openstack-ironic | 19:26 | |
*** markguz has quit IRC | 19:29 | |
*** markguz has joined #openstack-ironic | 19:35 | |
*** dprince has quit IRC | 19:37 | |
*** r-daneel has quit IRC | 19:39 | |
*** markguz_ has joined #openstack-ironic | 19:40 | |
*** r-daneel has joined #openstack-ironic | 19:40 | |
*** markguz_ has quit IRC | 19:40 | |
*** markguz_ has joined #openstack-ironic | 19:40 | |
*** markguz__ has joined #openstack-ironic | 19:42 | |
*** markguz__ has quit IRC | 19:42 | |
*** markguz has quit IRC | 19:42 | |
*** markguz has joined #openstack-ironic | 19:42 | |
*** markguz_ has quit IRC | 19:45 | |
*** racedo has joined #openstack-ironic | 19:47 | |
*** med_ has quit IRC | 19:50 | |
*** arnewiebalck_ has quit IRC | 20:04 | |
*** hjensas has quit IRC | 20:04 | |
*** dprince has joined #openstack-ironic | 20:05 | |
*** racedo has quit IRC | 20:21 | |
openstackgerrit | Jim Rollenhagen proposed openstack/ironic master: Use conductor group for hash ring calculations https://review.openstack.org/579684 | 20:22 |
openstackgerrit | Jim Rollenhagen proposed openstack/ironic master: Expose node.conductor_group in the REST API https://review.openstack.org/581391 | 20:22 |
jroll | so this goes back to what we originally had for the hash ring keys. I think it should be fine. | 20:23 |
needssleep | okay | 20:23 |
jroll | also solves dtantsur|afk's concerns with the driver API, though doesn't move it out of the dbapi | 20:23 |
jroll | and now I see waterloo suggested what I did :D | 20:24 |
jroll | great minds and all | 20:24 |
*** med_ has joined #openstack-ironic | 20:26 | |
*** med_ has quit IRC | 20:26 | |
*** med_ has joined #openstack-ironic | 20:27 | |
*** racedo has joined #openstack-ironic | 20:28 | |
*** rh-jelabarre has quit IRC | 20:29 | |
*** med_ has quit IRC | 20:30 | |
*** rh-jelabarre has joined #openstack-ironic | 20:34 | |
*** mjturek has quit IRC | 20:37 | |
*** r-daneel has quit IRC | 20:40 | |
*** spartacloud has joined #openstack-ironic | 20:41 | |
*** spartacloud has quit IRC | 20:45 | |
*** racedo has quit IRC | 20:49 | |
waterloo | jroll: :) and sorry about that | 20:50 |
*** arnewiebalck_ has joined #openstack-ironic | 20:55 | |
*** markguz has quit IRC | 21:00 | |
*** markguz has joined #openstack-ironic | 21:01 | |
*** trown is now known as trown|outtypewww | 21:04 | |
*** markguz_ has joined #openstack-ironic | 21:09 | |
*** markguz_ has quit IRC | 21:10 | |
*** markguz has quit IRC | 21:10 | |
*** markguz has joined #openstack-ironic | 21:11 | |
*** arnewiebalck_ has quit IRC | 21:11 | |
*** dprince has quit IRC | 21:11 | |
*** csmart[m] has quit IRC | 21:17 | |
*** jovial[m] has quit IRC | 21:17 | |
*** retr0h[m] has quit IRC | 21:17 | |
*** madhuri[m] has quit IRC | 21:17 | |
*** r-daneel has joined #openstack-ironic | 21:20 | |
*** e0ne has quit IRC | 21:27 | |
*** e0ne has joined #openstack-ironic | 21:27 | |
*** spartacloud has joined #openstack-ironic | 21:28 | |
*** e0ne has quit IRC | 21:30 | |
*** spartacloud has quit IRC | 21:33 | |
*** markguz_ has joined #openstack-ironic | 21:39 | |
*** figleaf is now known as edleafe | 21:41 | |
*** markguz has quit IRC | 21:42 | |
*** markguz_ has quit IRC | 21:44 | |
waterloo | jroll: i reviewed/commented. we should be able to land this early next week :) | 22:01 |
*** fried_rice is now known as efried | 22:09 | |
*** jovial[m] has joined #openstack-ironic | 22:11 | |
*** weshay is now known as weshay_PTO | 22:15 | |
*** racedo has joined #openstack-ironic | 22:27 | |
*** waterloo has quit IRC | 22:32 | |
*** tssurya has quit IRC | 22:52 | |
*** alex_xu has quit IRC | 22:53 | |
*** alex_xu has joined #openstack-ironic | 23:00 | |
*** racedo has quit IRC | 23:00 | |
*** madhuri[m] has joined #openstack-ironic | 23:16 | |
*** csmart[m] has joined #openstack-ironic | 23:16 | |
*** retr0h[m] has joined #openstack-ironic | 23:16 | |
*** r-daneel has quit IRC | 23:26 | |
*** r-daneel has joined #openstack-ironic | 23:29 | |
*** markguz has joined #openstack-ironic | 23:33 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!