| *** gyee has quit IRC | 00:03 | |
| *** livelace2 has quit IRC | 00:10 | |
| *** livelace2 has joined #openstack-ironic | 00:10 | |
| openstackgerrit | waleed mousa proposed openstack/ironic-python-agent master: Configuring mellanox interfaces using ironic python agent https://review.openstack.org/566544 | 00:17 |
|---|---|---|
| *** ijw has joined #openstack-ironic | 00:21 | |
| *** threestrands has quit IRC | 00:38 | |
| *** spartacloud has joined #openstack-ironic | 00:48 | |
| *** rcernin_ has joined #openstack-ironic | 00:56 | |
| *** bdodd has quit IRC | 00:56 | |
| *** rcernin has quit IRC | 00:58 | |
| *** phuongnh has joined #openstack-ironic | 01:04 | |
| *** phuongnh has quit IRC | 01:04 | |
| *** links has joined #openstack-ironic | 01:17 | |
| *** spartacloud has quit IRC | 01:18 | |
| *** jiapei has joined #openstack-ironic | 01:20 | |
| *** etingof has quit IRC | 01:22 | |
| openstackgerrit | Kaifeng Wang proposed openstack/ironic-python-agent master: Collect IPv6 address during introspection https://review.openstack.org/536136 | 01:23 |
| *** spartacloud has joined #openstack-ironic | 01:28 | |
| *** tiendc has joined #openstack-ironic | 01:31 | |
| *** stendulker has joined #openstack-ironic | 01:32 | |
| *** spartacloud has quit IRC | 01:32 | |
| *** etingof has joined #openstack-ironic | 01:35 | |
| *** ijw has quit IRC | 02:04 | |
| *** stendulker has quit IRC | 02:04 | |
| *** rloo has quit IRC | 02:13 | |
| *** gcb has joined #openstack-ironic | 02:16 | |
| *** alexpilotti has joined #openstack-ironic | 02:17 | |
| *** alexpilotti has quit IRC | 02:21 | |
| *** phuongnh has joined #openstack-ironic | 03:06 | |
| *** r-daneel has joined #openstack-ironic | 03:21 | |
| *** ijw has joined #openstack-ironic | 03:27 | |
| *** ijw has quit IRC | 03:32 | |
| *** phuongnh has quit IRC | 03:33 | |
| *** phuongnh has joined #openstack-ironic | 03:34 | |
| *** r-daneel has quit IRC | 03:39 | |
| *** bdodd has joined #openstack-ironic | 03:44 | |
| *** spartacloud has joined #openstack-ironic | 03:45 | |
| *** bdodd has quit IRC | 03:48 | |
| *** spartacloud has quit IRC | 03:49 | |
| *** bdodd has joined #openstack-ironic | 03:52 | |
| *** stendulker has joined #openstack-ironic | 03:57 | |
| *** zhangfei has joined #openstack-ironic | 04:06 | |
| *** hshiina has joined #openstack-ironic | 04:11 | |
| openstackgerrit | Nguyen Van Trung proposed openstack/ironic master: Update docs for troubleshooting with maintainance mode https://review.openstack.org/585126 | 04:13 |
| *** spartacloud has joined #openstack-ironic | 04:14 | |
| *** r-daneel has joined #openstack-ironic | 04:17 | |
| *** spartacloud has quit IRC | 04:24 | |
| *** luzC has quit IRC | 04:25 | |
| *** karimull has quit IRC | 04:25 | |
| *** karimull has joined #openstack-ironic | 04:25 | |
| *** luzC has joined #openstack-ironic | 04:28 | |
| *** ijw has joined #openstack-ironic | 04:33 | |
| *** phuongnh has quit IRC | 04:34 | |
| *** phuongnh has joined #openstack-ironic | 04:35 | |
| *** ijw has quit IRC | 04:38 | |
| *** e0ne has joined #openstack-ironic | 04:40 | |
| *** e0ne has quit IRC | 04:40 | |
| *** jaganathan has quit IRC | 04:42 | |
| *** jaganathan has joined #openstack-ironic | 04:42 | |
| *** honza has quit IRC | 04:47 | |
| *** honza has joined #openstack-ironic | 04:48 | |
| *** honza is now known as Guest56850 | 04:49 | |
| *** e0ne has joined #openstack-ironic | 04:53 | |
| *** e0ne has quit IRC | 04:55 | |
| *** ijw has joined #openstack-ironic | 05:04 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Fix for failure in cleaning https://review.openstack.org/580136 | 05:07 |
| *** pc_m has quit IRC | 05:09 | |
| *** ijw has quit IRC | 05:11 | |
| *** pc_m has joined #openstack-ironic | 05:13 | |
| *** tiendc has quit IRC | 05:32 | |
| *** tiendc has joined #openstack-ironic | 05:32 | |
| *** zhangfei has quit IRC | 05:43 | |
| *** mjura has joined #openstack-ironic | 05:44 | |
| *** zhangfei has joined #openstack-ironic | 05:56 | |
| *** rbartal has joined #openstack-ironic | 06:12 | |
| *** skazi has joined #openstack-ironic | 06:16 | |
| *** stendulker_ has joined #openstack-ironic | 06:18 | |
| *** stendulker has quit IRC | 06:21 | |
| *** zhangfei has quit IRC | 06:28 | |
| *** pcaruana has joined #openstack-ironic | 06:34 | |
| *** tssurya has joined #openstack-ironic | 06:43 | |
| openstackgerrit | Hironori Shiina proposed openstack/ironic master: snmp: Keep get_next method backward-compatible https://review.openstack.org/584647 | 06:46 |
| *** arnewiebalck_ has joined #openstack-ironic | 06:53 | |
| *** arnewiebalck_ has quit IRC | 06:57 | |
| *** stendulker has joined #openstack-ironic | 07:05 | |
| *** stendulker_ has quit IRC | 07:08 | |
| *** racedo has joined #openstack-ironic | 07:16 | |
| *** tesseract has joined #openstack-ironic | 07:18 | |
| openstackgerrit | Kaifeng Wang proposed openstack/ironic-inspector master: [WIP] Wrap rpc server into oslo.service https://review.openstack.org/584758 | 07:24 |
| *** ajya has joined #openstack-ironic | 07:30 | |
| *** zhangfei has joined #openstack-ironic | 07:31 | |
| *** phuongnh has quit IRC | 07:32 | |
| *** phuongnh has joined #openstack-ironic | 07:33 | |
| *** arnewiebalck_ has joined #openstack-ironic | 07:46 | |
| *** AlexeyAbashkin has joined #openstack-ironic | 07:50 | |
| *** mjura has quit IRC | 07:53 | |
| *** arnewiebalck_ has quit IRC | 07:53 | |
| *** rcernin_ has quit IRC | 07:56 | |
| *** stendulker_ has joined #openstack-ironic | 08:03 | |
| *** stendulker has quit IRC | 08:06 | |
| *** derekh has joined #openstack-ironic | 08:07 | |
| *** skazi has quit IRC | 08:07 | |
| *** rcernin_ has joined #openstack-ironic | 08:09 | |
| *** dougsz has joined #openstack-ironic | 08:10 | |
| *** gkadam has joined #openstack-ironic | 08:13 | |
| *** e0ne has joined #openstack-ironic | 08:15 | |
| openstackgerrit | Olivier Bourdon proposed openstack/ironic-lib stable/queens: Fix GPT bug with whole disk images https://review.openstack.org/579818 | 08:22 |
| *** dtantsur|afk is now known as dtantsur | 08:33 | |
| dtantsur | TheJulia: all current ironic-staging-drivers-core members agreed to your addition, so welcome :) | 08:35 |
| dtantsur | TheJulia: you can celebrate it by reviewing https://review.openstack.org/567902 ;) | 08:36 |
| patchbot | patch 567902 - ironic-staging-drivers - Remove classic drivers | 08:36 |
| etingof | morning Ironic o/ | 08:41 |
| arnewiebalck | Good morning Ironic! The API processes in our deployment have a memory footprint of 300MB RSS each. As the DB is only 50MB, I was wondering what that memory is used for .. any idea? | 08:42 |
| dtantsur | arnewiebalck: python just eating memory? :) hard to tell without profiling I guess. why don't you use it with wsgi? | 08:51 |
| arnewiebalck | dtantsur: It seems the process starts with 70MB and goes to 300MB upon first get all nodes request. | 08:52 |
| arnewiebalck | dtantsur: Moving to wsgi is on the list :) | 08:52 |
| arnewiebalck | dtantsur: Guess it makes sense to do that first and come back if the issue persists ... | 08:53 |
| arnewiebalck | dtantsur: I was just wondering what that scales with. | 08:53 |
| arnewiebalck | dtantsur: Like: no of nodes (DB size), no of reqs, ... | 08:54 |
| dtantsur | arnewiebalck: to me it should not change much, unless we're leaking memory somewhere | 08:54 |
| dtantsur | on my testing machine I have two wsgi processes around 110M each | 08:55 |
| dtantsur | I wonder if it could be oslo.messaging related | 08:55 |
| dtantsur | though listing nodes does not use RPC | 08:55 |
| *** mjura has joined #openstack-ironic | 08:56 | |
| *** priteau has joined #openstack-ironic | 08:56 | |
| arnewiebalck | dtantsur: I got a QA setup with much less nodes (50 rather than >1000), and there the RSS is also ~100MB | 09:00 |
| dtantsur | arnewiebalck: hmm, maybe it's cached hash ring that takes space? | 09:00 |
| dtantsur | though I can hardly imagine it taking 200M | 09:00 |
| arnewiebalck | dtantsur: So, the DB is 20x smaller, but the RSS footprint only a factor 3. | 09:01 |
| *** hshiina has quit IRC | 09:01 | |
| *** stendulker has joined #openstack-ironic | 09:01 | |
| arnewiebalck | dtantsur: The prod one has 600MB VIRT and 300MB RSS on each worker. And from what I see, it jumps on the first request. So, I was wondering if SQLAlchemy would get the data from the DB and “nicely” lay this out in memory ... | 09:03 |
| *** dmellado has quit IRC | 09:03 | |
| arnewiebalck | dtantsur: All I could get so far is that there are 200MB on the heap. | 09:04 |
| arnewiebalck | dtantsur: Let me try to move wsgi, which I wanted to do anyway, and then revisit the situation. | 09:05 |
| *** stendulker_ has quit IRC | 09:05 | |
| dtantsur | arnewiebalck: keep in mind that python may not return big chunks of memory back to linux even if it's free | 09:07 |
| dtantsur | I wonder if this is what's happening for you | 09:07 |
| arnewiebalck | dtantsur: Hmm, that would explain why the Python memory dump found only 50MB our of the 300MB. | 09:11 |
| *** stendulker_ has joined #openstack-ironic | 09:16 | |
| arnewiebalck | dtantsur: OTOH, I just triggered a python garbage collection on the API: didn’t change the memory footprint. | 09:16 |
| dtantsur | arnewiebalck: gc won't help in this case. it's just the way python manages memory | 09:17 |
| dtantsur | it allocates quite big pools and only returns to malloc those completely empty | 09:17 |
| arnewiebalck | dtantsur: Ah, ok. | 09:17 |
| *** stendulker has quit IRC | 09:19 | |
| *** stendulker_ has quit IRC | 09:35 | |
| *** stendulker_ has joined #openstack-ironic | 09:35 | |
| *** dsneddon has quit IRC | 09:36 | |
| *** oanson has quit IRC | 09:45 | |
| *** oanson has joined #openstack-ironic | 09:45 | |
| *** penick has quit IRC | 09:46 | |
| *** penick has joined #openstack-ironic | 09:49 | |
| *** betherly-afk is now known as betherly_ | 10:06 | |
| etingof | arnewiebalck, you may consider trying a memory profiler like guppy or muppy in hope to get hold of the source the leak... | 10:17 |
| arnewiebalck | etingof: Thanks! I tried, with little success. I did a memory dump with pyrasite, but that found only 50MB out of the 300MB in the Python process. | 10:18 |
| arnewiebalck | etingof: I was about to try with gdb directly, but now I probaly move to https first :) | 10:18 |
| arnewiebalck | etingof: Hoping the issue turns into a non-issue ;-) | 10:19 |
| etingof | arnewiebalck, that's the thing about profilers - they seem to tackle the problem differently thus reporting different things | 10:19 |
| arnewiebalck | etingof: The kernel is pretty consistent, though, and pmap says the process has 200MB on the heap. | 10:19 |
| *** phuongnh has quit IRC | 10:20 | |
| arnewiebalck | etingof: But, yeah, not sure if that memory dump I did was accurate or complete. | 10:20 |
| arnewiebalck | etingof: We’re preparing now the move to httpd, I’ll report back. | 10:21 |
| *** spartacloud has joined #openstack-ironic | 10:21 | |
| etingof | arnewiebalck, also, theoretically, some process memory can be allocated by extension modules so it may not show up as python objects... | 10:21 |
| arnewiebalck | etingof: What examples of such modules be? | 10:22 |
| arnewiebalck | etingof: What would … | 10:22 |
| etingof | arnewiebalck, database wrapper for example? | 10:23 |
| arnewiebalck | etingof: SQLAlchemy | 10:23 |
| etingof | e.g. python bindings to a database client | 10:23 |
| *** dmellado has joined #openstack-ironic | 10:24 | |
| openstackgerrit | vinay kumar muddu proposed openstack/ironic master: Documentation update of iLO BIOS settings https://review.openstack.org/585202 | 10:25 |
| arnewiebalck | etingof: shouldn’t pmap report it as such then? | 10:25 |
| *** spartacloud has quit IRC | 10:25 | |
| etingof | arnewiebalck, indeed, but not python-level memory profiler. that's what I am trying to say | 10:26 |
| *** e0ne has quit IRC | 10:27 | |
| arnewiebalck | etingof: Ah, I see. | 10:27 |
| openstackgerrit | Olivier Bourdon proposed openstack/ironic-lib stable/queens: Fix GPT bug with whole disk images https://review.openstack.org/579818 | 10:28 |
| *** arnewiebalck_ has joined #openstack-ironic | 10:29 | |
| arnewiebalck | dtantsur etingof : Let’s see what moving to httpd gives ... thx for your suggestions! | 10:32 |
| *** e0ne has joined #openstack-ironic | 10:32 | |
| *** arnewiebalck_ has quit IRC | 10:33 | |
| *** e0ne has quit IRC | 10:40 | |
| *** racedo has quit IRC | 10:40 | |
| *** stendulker has joined #openstack-ironic | 10:43 | |
| *** stendulker_ has quit IRC | 10:47 | |
| *** skazi has joined #openstack-ironic | 10:51 | |
| *** rcernin_ has quit IRC | 10:52 | |
| *** stendulker_ has joined #openstack-ironic | 11:00 | |
| *** stendulker has quit IRC | 11:03 | |
| *** stendulker_ has quit IRC | 11:04 | |
| *** alexpilotti has joined #openstack-ironic | 11:05 | |
| *** alexpilotti has quit IRC | 11:05 | |
| *** alexpilotti has joined #openstack-ironic | 11:06 | |
| dtantsur | etingof: we use pure-python mysql bindings because eventlet | 11:07 |
| dtantsur | I'd still bet on the free objects pools | 11:08 |
| etingof | ah, I see | 11:08 |
| *** alexpilo_ has joined #openstack-ironic | 11:08 | |
| *** alexpilotti has quit IRC | 11:11 | |
| openstackgerrit | Olivier Bourdon proposed openstack/ironic-lib stable/queens: Fix GPT bug with whole disk images https://review.openstack.org/579818 | 11:13 |
| *** racedo has joined #openstack-ironic | 11:16 | |
| *** gcb has quit IRC | 11:23 | |
| jroll | morning | 11:26 |
| *** dougsz has quit IRC | 11:31 | |
| *** dougsz has joined #openstack-ironic | 11:32 | |
| dtantsur | morning jroll | 11:32 |
| jroll | hey dtantsur :) | 11:35 |
| jroll | despite what zuul says, the conductor_group stuff seems ready | 11:35 |
| jroll | when you have a few :) | 11:35 |
| dtantsur | on my radar, just trying to wake up :) | 11:37 |
| *** zhangfei has quit IRC | 11:38 | |
| * jroll passes around coffee | 11:38 | |
| openstackgerrit | yolanda.robla proposed openstack/ironic master: WIP: Add documentation for BIOS settings https://review.openstack.org/585322 | 11:42 |
| *** r-daneel has quit IRC | 11:46 | |
| *** rh-jelabarre has joined #openstack-ironic | 11:47 | |
| *** serlex has joined #openstack-ironic | 11:49 | |
| *** e0ne has joined #openstack-ironic | 11:53 | |
| *** linkmark has joined #openstack-ironic | 11:54 | |
| TheJulia | good morning | 11:54 |
| *** Phuongnh has joined #openstack-ironic | 11:54 | |
| etingof | \o | 11:56 |
| *** tiendc has quit IRC | 11:56 | |
| *** e0ne has quit IRC | 11:58 | |
| *** MrBerns has quit IRC | 11:58 | |
| *** MrBerns has joined #openstack-ironic | 11:59 | |
| dtantsur | morning TheJulia | 11:59 |
| *** namnh has joined #openstack-ironic | 12:01 | |
| *** e0ne has joined #openstack-ironic | 12:07 | |
| *** MattMan has quit IRC | 12:08 | |
| *** MattMan has joined #openstack-ironic | 12:08 | |
| *** jiapei has quit IRC | 12:10 | |
| *** r-daneel has joined #openstack-ironic | 12:11 | |
| *** trown|outtypewww is now known as trown | 12:11 | |
| *** strigazi has quit IRC | 12:13 | |
| *** strigazi has joined #openstack-ironic | 12:14 | |
| *** r-daneel has quit IRC | 12:14 | |
| TheJulia | feels like we're starting to see a higher rate of failures for some of our tests :( | 12:18 |
| dtantsur | yep :( | 12:21 |
| *** MrBerns has quit IRC | 12:23 | |
| *** namnh has quit IRC | 12:24 | |
| *** MrBerns has joined #openstack-ironic | 12:24 | |
| *** namnh has joined #openstack-ironic | 12:24 | |
| *** namnh has quit IRC | 12:31 | |
| dtantsur | jroll: both patches LGTM, thanks | 12:33 |
| TheJulia | I just looked at the first one, looks great to me | 12:33 |
| TheJulia | I'm going to take a look at the second one | 12:33 |
| TheJulia | I've not really seen a pattern with the test failures yet | 12:33 |
| TheJulia | :\ | 12:33 |
| dtantsur | yeah, nothing specific :( | 12:33 |
| MrBerns | Hi guys, would someone be able to help me with my node deployment failure on bifrost, I am still having the parted issue from before: http://paste.openstack.org/show/iGyjBweDzfZLK5SOFipD/ | 12:34 |
| MrBerns | I am booted into the ramdisk atm if any extra debugging info is required | 12:34 |
| dtantsur | MrBerns: some context needed: whole disk or partition image, msdos vs gpt, etc | 12:36 |
| *** racedo has quit IRC | 12:37 | |
| MrBerns | dtantsur, I am not sure with regards to the above. I am just trying to boot a node I have enrolled in bifrost using the deploy-dynamic playbook. From what I can see in the IPA logs it runs the write_image.sh script to write the qcow2 (standard ubuntu cloudimage) file to disk, says its written it and immidiately prints the above error | 12:37 |
| MrBerns | From what I can tell its the whole disk but I don't know which partition table bifrost is using | 12:39 |
| dtantsur | MrBerns: msdos by default. but I'm suspecting that it's a partition image, so we end up without partition table.. | 12:39 |
| dtantsur | mind pasting a link to the image you're using? | 12:40 |
| openstackgerrit | Olivier Bourdon proposed openstack/ironic-python-agent master: Add configurable value for get-pip.py retrieval https://review.openstack.org/585337 | 12:41 |
| olivierb- | hello everyone. | 12:43 |
| MrBerns | dtantsur, Im not sure how to share the whole image, but I built it using the instructions in https://docs.openstack.org/bifrost/latest/user/troubleshooting.html (Thats the Coreos one) | 12:44 |
| MrBerns | I previously also tried using Centos and Ubuntu images with the same result | 12:44 |
| openstackgerrit | Olivier Bourdon proposed openstack/ironic-lib stable/queens: Fix GPT bug with whole disk images https://review.openstack.org/579818 | 12:45 |
| *** racedo has joined #openstack-ironic | 12:45 | |
| dtantsur | MrBerns: no, I meant the ubuntu image, is it something like https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img ? | 12:45 |
| olivierb- | TheJulia trying to find a way to fix https://review.openstack.org/579818 which goes further now that I have added lines with PIP_GET_PIP_URL='https://bootstrap.pypa.io/3.2/get-pip.py' the same errors occurs a bit further down the path and therefore I made https://review.openstack.org/585337 | 12:45 |
| patchbot | patch 579818 - ironic-lib (stable/queens) - Fix GPT bug with whole disk images | 12:45 |
| patchbot | patch 585337 - ironic-python-agent - Add configurable value for get-pip.py retrieval | 12:45 |
| olivierb- | what do you think ? | 12:46 |
| TheJulia | olivierb-: let me get something to eat and I'll dig. I'm just finishing up an email at the moment | 12:46 |
| olivierb- | of course, have a nice and calm breakfast ;-) | 12:46 |
| MrBerns | dtantsur, Oh pardon yes thats the one | 12:47 |
| MrBerns | I converted it to qemu using 'qemu-img convert -f raw -O' | 12:47 |
| MrBerns | Not qemu, qcow2 :\ | 12:48 |
| *** rloo has joined #openstack-ironic | 12:51 | |
| dtantsur | that's a weird image, hmm | 12:53 |
| dtantsur | aha, it's a GPT image | 12:54 |
| TheJulia | Okay, PTL hat done for the morning | 12:59 |
| rloo | good morning ironic'ers | 13:00 |
| dtantsur | so it probably fails at https://github.com/openstack/ironic-lib/blob/master/ironic_lib/disk_utils.py#L821 because parted cannot recognize the GPT partition. mmmm... | 13:00 |
| dtantsur | morning rloo | 13:00 |
| *** Phuongnh has quit IRC | 13:00 | |
| rloo | morning dtantsur, TheJulia :) | 13:00 |
| rloo | did any of you look to see why https://review.openstack.org/#/c/579684/ is failing? | 13:01 |
| patchbot | patch 579684 - ironic - Use conductor group for hash ring calculations | 13:01 |
| TheJulia | rloo: yeah, looks like the port change never worked or timeed out. Unrelated to the change its self | 13:01 |
| rloo | Ok, so recheck? | 13:01 |
| TheJulia | rloo: if it is not already actively running a recheck, yes | 13:01 |
| rloo | TheJulia: ah, yeah, I see that jroll has rechecked. (i need to wake up) | 13:02 |
| MrBerns | dtantsur, So I need to use an image that uses msdos? | 13:03 |
| dtantsur | MrBerns: you should be fine with GPT, I don't get why it fails.. | 13:04 |
| MrBerns | Is there anything else IPA should be doing before trying to create that partition? Apart from writing the qcow image to disk that is. Perhaps some step is getting skipped | 13:05 |
| openstackgerrit | Olivier Bourdon proposed openstack/ironic-lib stable/queens: Fix GPT bug with whole disk images https://review.openstack.org/579818 | 13:07 |
| *** ianychoi has quit IRC | 13:08 | |
| TheJulia | olivierb-: LGTM, I suspect we may need to end up squashing the changes together for stable branches, I guess lets wait and see a little bit. | 13:08 |
| jroll | morning everyone that showed up since I last spoke :) | 13:08 |
| *** ianychoi has joined #openstack-ironic | 13:08 | |
| jroll | dtantsur: thanks for the class-level use_groups mention, I knew there was something but was blanking on it | 13:09 |
| * jroll does follow ups | 13:09 | |
| olivierb- | TheJulia in fact I was wrong in adding the variable to ironic-lib patch | 13:09 |
| olivierb- | first because it is not part of the backport of this very specific bug | 13:09 |
| dtantsur | MrBerns: you said you're in the ramdisk now, right? if you try $ parted -s -m /dev/sdc unit MiB print, does it fail the same way? is /dev/sdc actually the correct disk? | 13:09 |
| TheJulia | wow, MrBerns issue is kind of bizzar :( | 13:09 |
| olivierb- | second because I was wrong assuming that the 1st get-pip.py retrieval was failing | 13:10 |
| olivierb- | only the one when building tinyipa is failing somehow | 13:10 |
| MrBerns | Error: /dev/sda: unrecognised disk label | 13:10 |
| olivierb- | I reverted https://review.openstack.org/579818 to your previous version | 13:10 |
| patchbot | patch 579818 - ironic-lib (stable/queens) - Fix GPT bug with whole disk images | 13:10 |
| TheJulia | olivierb-: I thought it was earlier on, but with tinyipa we also get to run an older grumpier python, so that change is totally appropriate I think | 13:11 |
| *** arnewiebalck_ has joined #openstack-ironic | 13:11 | |
| *** cdearborn has joined #openstack-ironic | 13:11 | |
| olivierb- | and will backport https://review.openstack.org/585337 to stable versions when consensus is reached ;-). thanks a lot | 13:11 |
| patchbot | patch 585337 - ironic-python-agent - Add configurable value for get-pip.py retrieval | 13:11 |
| MrBerns | But then it prints the rest of the command output | 13:11 |
| dtantsur | MrBerns: what about sfdisk -l? | 13:12 |
| dtantsur | oh, hmm. and what's this output? | 13:12 |
| dtantsur | I wonder if it's the problem olivierb- is trying to solve | 13:12 |
| MrBerns | Im on the graphical console atm so theres no easy way for me to paste this output, but it looks like a standard fdisk output sda looks fine and there are no partitions on it | 13:13 |
| TheJulia | dtantsur: I think they ar edifferent | 13:13 |
| TheJulia | Oo | 13:13 |
| dtantsur | MrBerns: is it sda or sdc? your paste mentions sdc | 13:13 |
| dtantsur | TheJulia: okay. I got triggered by "GPT" "whole disk images" | 13:14 |
| MrBerns | Pardon, should clarify, Ive been having this issue for a couple of days, the node im currently booted into is sda, the paste from yday was a different node and is sdc | 13:14 |
| MrBerns | I just replaced the qcow image again and tried to boot the node from yday and now it worked :S | 13:15 |
| dtantsur | O_o | 13:15 |
| dtantsur | do you have automated cleaning enabled? maybe something remaining on nodes confuse IPA? | 13:15 |
| openstackgerrit | Jim Rollenhagen proposed openstack/ironic master: Simplify hash ring tests https://review.openstack.org/585346 | 13:15 |
| openstackgerrit | Jim Rollenhagen proposed openstack/ironic master: Be more precise with conductor group API tests https://review.openstack.org/585347 | 13:15 |
| *** arnewiebalck_ has quit IRC | 13:15 | |
| MrBerns | No cleaning is disabled for now as we've had a load of other problems with that | 13:16 |
| MrBerns | Let me try and boot another node | 13:16 |
| *** VisBits has joined #openstack-ironic | 13:16 | |
| dtantsur | rloo: no pressure, but I thin we're waiting for your review on the conductor group stuff :) | 13:18 |
| rloo | dtantsur: looking now :) | 13:18 |
| *** jaypipes has quit IRC | 13:20 | |
| * TheJulia needs a caffeine IV | 13:20 | |
| rloo | dtantsur: +A'd | 13:21 |
| dtantsur | \o/ | 13:21 |
| rloo | now it is up to zuul | 13:21 |
| rloo | no pressure zuul... :D | 13:22 |
| TheJulia | heh | 13:22 |
| rloo | dtantsur, TheJulia: you good with https://review.openstack.org/#/c/581391? Or are you waiting for me to review that? | 13:22 |
| patchbot | patch 581391 - ironic - Expose node.conductor_group in the REST API | 13:22 |
| TheJulia | I'm good with it if your not up for reviewing it | 13:23 |
| rloo | I can review it -- was wondering if you wanted me to do so or not :) I'll look | 13:23 |
| * TheJulia loads the zuul dahsboard with the string "ironic" and just keeps it open to keep an eye out for failing jobs | 13:23 | |
| dtantsur | rloo: I don't insist :) | 13:24 |
| openstackgerrit | Derek Higgins proposed openstack/ironic master: [TEST] [DO NOT MERGE] Nothing to see here https://review.openstack.org/487118 | 13:25 |
| TheJulia | Re: ramdisk spec, it has 2x+2, code is presently sitting with a -2 until the spec is approved. I'm happy to push a follow-up to the spec later today, since we seem to have approached consensus. | 13:26 |
| dtantsur | TheJulia: I thought it's fair to wait for other people to review. if other cores (jroll, sambetts, vdrok, whoever is around) are not up for reviewing it, I'm fine with approving. | 13:27 |
| TheJulia | That is fair | 13:28 |
| jroll | I'm fine reviewing it soon or after it lands, either way | 13:28 |
| jroll | I've realized my needs with that are slightly different, we're burning grub and ipxe into the disk and then netbooting from there | 13:29 |
| jroll | so need to figure out a way to tweak that | 13:29 |
| TheJulia | mgoddard is also not around today it seems :\ | 13:30 |
| TheJulia | jroll: interesting... | 13:30 |
| TheJulia | well, if we still write the ipxe template, all the dhcp server needs to do is say "here, go read this url for your script" | 13:30 |
| jroll | yep | 13:31 |
| MrBerns | Ok well it seems my issue is resolved now, thanks all | 13:31 |
| dtantsur | jroll: maybe similar to OVB, which uses an iPXE image as the instance image | 13:31 |
| jroll | TheJulia: I'd probably burn that part in as well, so we can totally remove dhcp | 13:31 |
| dtantsur | MrBerns: you're welcome :) I like when issues resolve on their own (I don't like when I don't understand them though) | 13:31 |
| TheJulia | jroll: vmedia booting? | 13:31 |
| jroll | TheJulia: no, just boot ipxe from disk and chain-load to an image from there | 13:32 |
| TheJulia | interesting | 13:32 |
| dtantsur | sounds like another deploy interface? :D | 13:32 |
| jroll | grub and ipxe are part of the trust chain here | 13:32 |
| jroll | so we burn them to disk in the "factory" before sending it off to the site | 13:33 |
| * jroll should write this long-form | 13:33 | |
| TheJulia | Yeah, I was pondering that for arm actually | 13:33 |
| TheJulia | but not pre-burnt on the node | 13:33 |
| openstackgerrit | yolanda.robla proposed openstack/ironic master: WIP: Add documentation for BIOS settings https://review.openstack.org/585322 | 13:37 |
| *** skazi has quit IRC | 13:37 | |
| rloo | jroll: where do you deal with notification updates, and doc updates (for conductor_groups)? | 13:38 |
| jroll | rloo: ugh, forgot about notifications. docs are still on my todo list | 13:38 |
| *** dprince has joined #openstack-ironic | 13:38 | |
| jroll | I can do notifications quickly as a follow up? | 13:38 |
| TheJulia | jroll: yes | 13:39 |
| rloo | yeah, i'm good if notifications are done soon. +A'd | 13:39 |
| jroll | I've never done notifications, is it just adding it to the notification object? | 13:39 |
| jroll | and bumping that version | 13:39 |
| rloo | jroll: it is a PITB. you add and bump versions in all payloads that inherit. and in the doc too. | 13:40 |
| TheJulia | jroll: basically that is correct, decorator, versions, and then stuff past that including doc | 13:40 |
| *** bdodd has quit IRC | 13:40 | |
| jroll | sheesh | 13:40 |
| jroll | thanks | 13:40 |
| TheJulia | jroll: it goes pretty quick | 13:41 |
| rloo | yeah, it isn't hard. just tedious :) | 13:41 |
| openstackgerrit | Derek Higgins proposed openstack/ironic master: [TEST] [DO NOT MERGE] Nothing to see here https://review.openstack.org/487118 | 13:41 |
| rloo | jroll: isn't there a client patch? | 13:42 |
| TheJulia | rloo: dmitry wrote it | 13:42 |
| rloo | different topic? I'll look for it... | 13:42 |
| jroll | rloo: https://review.openstack.org/584901 | 13:42 |
| patchbot | patch 584901 - python-ironicclient - Add support for conductor groups | 13:42 |
| TheJulia | it was the only outstanding python-ironicclient patch as of yesterday :) | 13:42 |
| rloo | Thx jroll. WOW, is that how we're supposed to use topics now? | 13:43 |
| rloo | 2001795/23117/conductor-group :) | 13:43 |
| TheJulia | i hope not | 13:43 |
| jroll | rloo: um, idk | 13:43 |
| jroll | I see a few like that | 13:43 |
| jroll | though may just be a dmitry thing | 13:44 |
| TheJulia | our docs say story/story# | 13:44 |
| jroll | the task isn't helpful if we actually break up the stories :) | 13:44 |
| rloo | interesting. but it isn't useful if it doesn't connect with the other patches 'for that topic' | 13:44 |
| dtantsur | TheJulia, jroll, yeah, it's how I do it | 13:44 |
| dtantsur | I expected gerrit to replace it like it used to with LP | 13:44 |
| rloo | i thought the idea of 'topic' was so you could get all the patches related to that topic. | 13:44 |
| dtantsur | apparently it's yet another missing feature | 13:44 |
| rloo | OH. Missing feature. Sigh. | 13:44 |
| dtantsur | the problem is that topics are derived from local branches | 13:45 |
| dtantsur | and I use this schema for local branches. which you never noticed because gerrit was replacing them with bug/XXX :) | 13:45 |
| *** bdodd has joined #openstack-ironic | 13:45 | |
| dtantsur | now I'll have to remember to do $ git review -t story/XXX | 13:45 |
| rloo | dtantsur: ah, the truth comes out :) | 13:46 |
| rloo | well, the good thing about topics is you can hand-edit them later. | 13:46 |
| dtantsur | right | 13:46 |
| openstackgerrit | Derek Higgins proposed openstack/ironic master: Fix the list of irrelevant-files https://review.openstack.org/585359 | 13:49 |
| jroll | alias gr='git review -t $(git branch | sed -e 's/([0-9]+)\/[0-9]+\/.*/story\/$1/')' | 13:50 |
| jroll | or something :P | 13:50 |
| dtantsur | not bad | 13:50 |
| *** bdodd has quit IRC | 13:54 | |
| *** r-daneel has joined #openstack-ironic | 14:02 | |
| openstackgerrit | Jim Rollenhagen proposed openstack/ironic master: Add conductor_group to node notifications https://review.openstack.org/585371 | 14:03 |
| *** MrBerns has quit IRC | 14:03 | |
| jroll | ^ there's notification stuffs rloo | 14:04 |
| *** baha has joined #openstack-ironic | 14:04 | |
| rloo | jroll: what, moar stuff to review? :) | 14:04 |
| jroll | yep! | 14:04 |
| jroll | and two easy fixups in front, now you have to look at those, muahaha | 14:04 |
| rloo | b* eautiful | 14:05 |
| jroll | :) | 14:05 |
| *** mjura has quit IRC | 14:05 | |
| olivierb- | TheJulia I just found about https://review.openstack.org/#/c/583520/ which does not seem to fix the issue but goes way further | 14:06 |
| patchbot | patch 583520 - ironic-python-agent - Pin get-pip.py to 3.2 | 14:06 |
| *** hamzy_ has quit IRC | 14:07 | |
| dtantsur | folks, any ideas where to put information about reset_interfaces feature? we don't seem to have much on changing drivers | 14:16 |
| rloo | dtantsur: i think we have two? main pages about drivers. looking... | 14:17 |
| *** mjturek has joined #openstack-ironic | 14:17 | |
| rloo | dtantsur: well, this seems to be the main page: https://docs.openstack.org/ironic/latest/admin/drivers.html | 14:18 |
| TheJulia | dtantsur: seems like an admin-ish task | 14:19 |
| rloo | dtantsur: i think the other page i'm thinking of is the one that had a table with classic-driver -> interfaces, but that is probably gone now. | 14:19 |
| rloo | dtantsur: so that page, add another section/link? | 14:19 |
| rloo | where is our description of the diff hardware interfaces? | 14:20 |
| dtantsur | yeah, this is my thought as well | 14:20 |
| dtantsur | in the install guide | 14:20 |
| rloo | dtantsur: OH. hmm. | 14:20 |
| rloo | dtantsur: I think it belongs in the admin, cuz you only want to do this (change hw type) after it has been set | 14:21 |
| rloo | dtantsur: maybe in the admin, add a new section, and in the new section, link to https://docs.openstack.org/ironic/latest/install/enabling-drivers.html | 14:22 |
| rloo | (and vice versa I guess) | 14:22 |
| dtantsur | yeah, good idea | 14:23 |
| TheJulia | dtantsur: how much longer are you going to be around today? | 14:24 |
| rloo | dtantsur: i'm good with approving this, with a follow up. wdyt? https://review.openstack.org/#/c/576717/ | 14:24 |
| patchbot | patch 576717 - ironic-specs - Add Boot from Ramdisk spec | 14:24 |
| *** bdodd has joined #openstack-ironic | 14:25 | |
| *** rbartal has quit IRC | 14:27 | |
| *** links has quit IRC | 14:31 | |
| TheJulia | olivierb-: looks like your tinyipa change on master is failing :( | 14:34 |
| olivierb- | TheJulia note sure to understand what you mean as https://review.openstack.org/585337 did not finish CI yet | 14:36 |
| patchbot | patch 585337 - ironic-python-agent - Add configurable value for get-pip.py retrieval | 14:36 |
| TheJulia | olivierb-: I can see it on http://zuul.openstack.org | 14:36 |
| olivierb- | anyways this is the 1st part of it because I need to change the env variables to effectively retrieve 3.2 version instead of latest | 14:36 |
| olivierb- | TheJulia keep forgetting about this and that I do not have to way to see some results | 14:37 |
| dtantsur | rloo++ | 14:37 |
| dtantsur | TheJulia: for 2 hours at least. why? | 14:37 |
| TheJulia | dtantsur: about to post... if the unit tests don't fail... the fix I put together for markguz | 14:38 |
| TheJulia | instad of adding a new config parameter | 14:39 |
| rloo | dtantsur: you can remove your -2 from https://review.openstack.org/#/c/568940/ | 14:39 |
| patchbot | patch 568940 - ironic - ramdisk deploy driver | 14:39 |
| TheJulia | ugh, 2 failed tests | 14:39 |
| dtantsur | sure | 14:39 |
| *** markguz has joined #openstack-ironic | 14:43 | |
| openstackgerrit | Aleksandra Bezborodova proposed openstack/sushy master: Add a virtual media resource https://review.openstack.org/570810 | 14:43 |
| *** markguz_ has joined #openstack-ironic | 14:49 | |
| *** rpioso|afk is now known as rpioso | 14:49 | |
| rpioso | Good morning | 14:49 |
| TheJulia | good morning rpioso | 14:50 |
| rpioso | TheJulia: o/ | 14:50 |
| *** hamzy_ has joined #openstack-ironic | 14:50 | |
| *** markguz has quit IRC | 14:52 | |
| *** hamzy_ has quit IRC | 14:52 | |
| *** hamzy_ has joined #openstack-ironic | 14:52 | |
| openstackgerrit | Merged openstack/ironic-specs master: Add Boot from Ramdisk spec https://review.openstack.org/576717 | 14:53 |
| openstackgerrit | Ilya Etingof proposed openstack/ironic master: Add Redfish inspect interface https://review.openstack.org/449425 | 14:55 |
| NobodyCam | Good Morning Ironic'ers | 14:57 |
| rpioso | NobodyCam: Good morning :) | 14:57 |
| dtantsur | morning NobodyCam | 14:57 |
| NobodyCam | Morning rpioso and dtantsur :) | 14:58 |
| *** arnewiebalck_ has joined #openstack-ironic | 14:59 | |
| *** MrBerns has joined #openstack-ironic | 15:02 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Change PXE logic to always link macs with UEFI https://review.openstack.org/578959 | 15:02 |
| TheJulia | dtantsur: ^^^ | 15:02 |
| dtantsur | great! will get to it soon | 15:02 |
| TheJulia | dtantsur: thanks | 15:03 |
| *** arnewiebalck_ has quit IRC | 15:04 | |
| *** jcoufal has joined #openstack-ironic | 15:04 | |
| *** rcernin_ has joined #openstack-ironic | 15:04 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: ramdisk deploy driver https://review.openstack.org/568940 | 15:10 |
| TheJulia | dtantsur: added reno ^^^ | 15:11 |
| * TheJulia is unsure of how verbose to be, but should begin on doc and tempest test later today | 15:11 | |
| dtantsur | cool. cannot promise to get this one today, but likely tomorrow | 15:11 |
| TheJulia | works for me | 15:11 |
| TheJulia | I'm going to step away and go get some lunch, bbiab | 15:14 |
| openstackgerrit | Aija Jaunteva proposed openstack/sushy-tools master: Add BIOS resource support https://review.openstack.org/580609 | 15:15 |
| dtantsur | https://www.openstack.org/summit/berlin-2018/vote-for-speakers | 15:21 |
| *** MrBerns2 has joined #openstack-ironic | 15:23 | |
| etingof | dtantsur, does task 10389 look plausible? -- https://storyboard.openstack.org/#!/story/1526753 | 15:24 |
| dtantsur | etingof: a bit more precisely "Add the redfish virtual media boot interface" | 15:24 |
| openstackgerrit | yolanda.robla proposed openstack/ironic master: WIP: Add documentation for BIOS settings https://review.openstack.org/585322 | 15:25 |
| *** MrBerns has quit IRC | 15:27 | |
| dtantsur | time for voting requests :) https://www.openstack.org/summit/berlin-2018/vote-for-speakers#/21993 | 15:27 |
| etingof | dtantsur, that's the title, right? are the notes good enough? | 15:27 |
| dtantsur | ah, notes | 15:28 |
| dtantsur | etingof: yep, the notes are correct | 15:29 |
| etingof | that seems to be quite a large piece of work | 15:29 |
| *** rcernin_ has quit IRC | 15:30 | |
| dtantsur | I'd say medium :) | 15:30 |
| *** pcaruana has quit IRC | 15:32 | |
| *** MrBerns3 has joined #openstack-ironic | 15:36 | |
| *** hongbin has joined #openstack-ironic | 15:36 | |
| *** gyee has joined #openstack-ironic | 15:37 | |
| *** MrBerns2 has quit IRC | 15:38 | |
| hongbin | folks, if you find your gate break due to the recent landed validation logic in neutron, you might consider merging this fix: https://review.openstack.org/#/c/583990/ | 15:38 |
| patchbot | patch 583990 - ironic-tempest-plugin - Correct the usage on listing neutron ports | 15:38 |
| dtantsur | TheJulia: ^^^ | 15:39 |
| * jroll is on it | 15:39 | |
| jroll | thanks hongbin! | 15:39 |
| hongbin | jroll: np | 15:40 |
| * dtantsur goes to this department store from time to time, might as well ask how their ironic is doing :D https://www.openstack.org/summit/berlin-2018/vote-for-speakers#/22595 | 15:40 | |
| * TheJulia returs with lunch | 15:43 | |
| *** arnewiebalck_ has joined #openstack-ironic | 15:53 | |
| openstackgerrit | Jim Rollenhagen proposed openstack/ironic master: Add conductor_group to node notifications https://review.openstack.org/585371 | 15:53 |
| openstackgerrit | Jim Rollenhagen proposed openstack/ironic master: Add deploy_step to NodePayload.SCHEMA https://review.openstack.org/585413 | 15:54 |
| jroll | ^ rloo, took care of the deploy_step thing for you | 15:54 |
| *** racedo has quit IRC | 15:55 | |
| rloo | thx jroll. it was also missing in the notifications doc; i thought i had reviewed that. we should just automate that doc anyway. one day... | 15:55 |
| *** arnewiebalck_ has quit IRC | 15:58 | |
| TheJulia | wheeee http://logs.openstack.org/84/579684/16/gate/ironic-tempest-dsvm-ipa-wholedisk-agent_ipmitool-tinyipa-multinode/557866f/logs/subnode-2/screen-n-cpu.txt.gz#_Jul_24_15_16_58_072284 | 16:06 |
| TheJulia | http://logs.openstack.org/84/579684/16/gate/ironic-tempest-dsvm-ipa-wholedisk-agent_ipmitool-tinyipa-multinode/557866f/job-output.txt.gz#_2018-07-24_15_17_24_808805 | 16:10 |
| *** serlex has quit IRC | 16:10 | |
| *** e0ne has quit IRC | 16:10 | |
| openstackgerrit | Olivier Bourdon proposed openstack/ironic-python-agent master: Add configurable value for get-pip.py retrieval https://review.openstack.org/585337 | 16:12 |
| *** trown is now known as trown|lunch | 16:16 | |
| *** dsneddon has joined #openstack-ironic | 16:17 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Add documentation for changing node's hardware type https://review.openstack.org/585463 | 16:21 |
| dtantsur | TheJulia, rloo ^^^ | 16:21 |
| *** lucasagomes is now known as lucas-afk | 16:24 | |
| rloo | dtantsur: NOOOOO, not documentation! | 16:25 |
| dtantsur | rloo: documentation FOR DRIVERS! | 16:25 |
| dtantsur | everyone's favourite | 16:25 |
| rloo | dtantsur: i'm not going to get to that this week. am only allocating time for doing stuff that is needed this week. | 16:25 |
| dtantsur | understandable. I just did it right now because otherwise I'll forget | 16:26 |
| rloo | dtantsur: i think the fav doc is the nonexisting ones for reference architecture guide :D | 16:26 |
| rloo | dtantsur: yeah, understand and good that you wrote that doc! | 16:26 |
| dtantsur | rloo: it's not completely non-existing, we have one case covered :) | 16:26 |
| rloo | dtantsur: heh, people are waiting with anticipation for the next chapter! | 16:27 |
| *** jaganathan has quit IRC | 16:27 | |
| * dtantsur too :) | 16:39 | |
| *** dtantsur is now known as dtantsur|afk | 16:39 | |
| dtantsur|afk | see you tomorrow | 16:39 |
| *** mjura has joined #openstack-ironic | 16:39 | |
| *** markguz has joined #openstack-ironic | 16:40 | |
| *** markguz_ has quit IRC | 16:43 | |
| *** tssurya has quit IRC | 16:43 | |
| TheJulia | EEK! Documentation! | 16:49 |
| * TheJulia hides under the piles of code | 16:49 | |
| *** derekh has quit IRC | 16:55 | |
| jroll | TheJulia: coincidentally another person on my team made this today. high-level boot flow that we're aiming for: https://imgur.com/Yfp2jFr.png | 16:57 |
| jroll | ipxe is on disk, the encryption key for the "host encrypted key" is in the TPM, the decrypted key is the client's private key for the mutual TLS auth to get the ramdisk | 17:00 |
| *** dougsz has quit IRC | 17:02 | |
| TheJulia | jroll: doing your own binary signing? | 17:04 |
| jroll | TheJulia: yeah, that process is all outside of the control plane things | 17:05 |
| *** mgoddard has joined #openstack-ironic | 17:06 | |
| TheJulia | jroll: would... say... the ipxe api surface that pas-ha proposed + security interface to handle back-end key/cert validation in some of the process be useful? | 17:07 |
| jroll | TheJulia: maybe? | 17:07 |
| jroll | if I was going to POC this (I am), what I would add is: | 17:08 |
| jroll | * a driver that boots straight to disk, no agent, with cleaning disabled | 17:08 |
| jroll | * said driver would also write/symlink the image to a known location, by node uuid, which the http server that does mTLS can serve from | 17:08 |
| jroll | * a way to store the encrypted key data | 17:09 |
| jroll | * a way to fetch the encrypted key data as plain text (not json etc), auth'd by IP/MAC/something else | 17:09 |
| jroll | (as iPXE would fetch that URL and set a variable to the entire response body) | 17:09 |
| jroll | I think that's it | 17:10 |
| jroll | but wanted to start the discussion to see if there's a better way to upstream this | 17:10 |
| TheJulia | ++ | 17:10 |
| *** AlexeyAbashkin has quit IRC | 17:11 | |
| TheJulia | I think that could work, keep a downstream driver that does $workflow magic needed, altough maybe that becomes custom deploy templates | 17:12 |
| jroll | yeah, will noodle about it | 17:14 |
| jroll | thanks :) | 17:14 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Ramdisk deploy driver doc https://review.openstack.org/585486 | 17:17 |
| *** tesseract has quit IRC | 17:17 | |
| TheJulia | Speaking of doc... *ducks* | 17:18 |
| *** ajya has quit IRC | 17:18 | |
| olivierb- | I know CI is very crowded but my last job is hanging there queued for the last 1h10mn or so :-( any reason I should worry about ? | 17:20 |
| TheJulia | olivierb-: this time of a cycle it ca take 2-4 hours to really get started | 17:24 |
| olivierb- | ah ok, it was so quick ealier today | 17:24 |
| clarkb | it is a queue, changes pushed first get tested first (roughly) | 17:24 |
| olivierb- | TheJulia thx for the info, will be patient then | 17:24 |
| clarkb | if you look at the check queue there are 214 changes in line | 17:25 |
| clarkb | and we are testing about 1/3 of them with the resources available. As those clear out the changes queued next will run tests | 17:25 |
| olivierb- | clarkb I understand thx for answering | 17:26 |
| clarkb | also the gate has priority so nodes are assigned to those jobs first | 17:26 |
| *** mjturek_ has joined #openstack-ironic | 17:28 | |
| *** mjturek has quit IRC | 17:29 | |
| *** e0ne has joined #openstack-ironic | 17:30 | |
| TheJulia | yeah, need to understand why we're starting to see sporadic failures on nova multinode... :\ | 17:30 |
| *** trown|lunch is now known as trown | 17:37 | |
| *** dprince has quit IRC | 17:42 | |
| openstackgerrit | Michael Turek proposed openstack/ironic master: Pass prep_boot_part_uuid to install_bootloader for ppc64* partition images https://review.openstack.org/579159 | 17:49 |
| mjturek_ | dtantsur|afk sorry for lagging on adding a reno ^ | 17:51 |
| *** mjturek_ is now known as mjturek | 17:51 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-tempest-plugin master: WIP: Ramdisk tempest test https://review.openstack.org/585495 | 17:53 |
| TheJulia | It has been WAY too long since I've changed anything in tempest | 17:54 |
| *** hamzy_ has quit IRC | 17:55 | |
| mjturek | TheJulia: is this your gate status from the IronicWhiteBoard? "Master branch: golden, stable may have issues with get-pip.py" | 17:55 |
| TheJulia | mjturek: yeah | 17:56 |
| mjturek | I'm hitting a get-pip issue here https://review.openstack.org/#/c/579159/ | 17:56 |
| patchbot | patch 579159 - ironic - Pass prep_boot_part_uuid to install_bootloader for... | 17:56 |
| mjturek | wondering if it is definitely related | 17:56 |
| mjturek | let me grab a timestamp | 17:56 |
| mjturek | whoops wrong patch | 17:56 |
| mjturek | https://review.openstack.org/#/c/582962/6 | 17:56 |
| patchbot | patch 582962 - ironic-python-agent - Install grub to PReP partition when prep_boot_part... | 17:56 |
| *** dprince has joined #openstack-ironic | 17:57 | |
| mjturek | timestamp http://logs.openstack.org/62/582962/6/check/ipa-tempest-dsvm-partition-bios-ipmi-direct-tinyipa-src/7d7a351/logs/devstacklog.txt.gz#_2018-07-23_15_10_18_205 | 17:57 |
| *** jaganathan has joined #openstack-ironic | 17:57 | |
| TheJulia | sure sounds like it is | 17:58 |
| mjturek | yaaaay - don't recheck yet I assume? | 17:58 |
| openstackgerrit | Mark Goddard proposed openstack/networking-generic-switch master: Add a netmiko device driver for Juniper switches https://review.openstack.org/530537 | 17:58 |
| TheJulia | https://review.openstack.org/#/c/585337/ | 17:59 |
| patchbot | patch 585337 - ironic-python-agent - Add configurable value for get-pip.py retrieval | 17:59 |
| *** ijw has joined #openstack-ironic | 18:00 | |
| mjturek | looking | 18:00 |
| jroll | erm | 18:05 |
| jroll | get-pip fails because uname doesn't work in the chroot | 18:05 |
| jroll | any reason we don't just fix that? | 18:05 |
| mjturek | jroll so just add uname deps to the chroot you think? | 18:06 |
| jroll | mjturek: well, idk why uname isn't working, but yeah | 18:07 |
| jroll | clark suggested bind-mounting /etc | 18:07 |
| jroll | but maybe there's an easier way to make it work on tinycorelinux | 18:07 |
| *** ijw has quit IRC | 18:08 | |
| TheJulia | uname should work in a chroot vi syscalls :\ | 18:10 |
| jroll | oh wait, we mock uname in that build? | 18:12 |
| jroll | https://github.com/openstack/ironic-python-agent/blob/dd69ef3cd59e2e78f58dcedba106b9d865395c36/imagebuild/tinyipa/build_files/fakeuname | 18:12 |
| jroll | lol | 18:12 |
| jroll | maybe that just needs to be earlier in the build | 18:12 |
| jroll | specifically https://github.com/openstack/ironic-python-agent/blob/80be07ae791980a1c444b3b0d685775c1688ca34/imagebuild/tinyipa/build-tinyipa.sh#L111 | 18:13 |
| TheJulia | wow | 18:13 |
| TheJulia | yeah, seems like it | 18:13 |
| jroll | urgh, it's before we run get-pip.py | 18:13 |
| *** e0ne has quit IRC | 18:14 | |
| TheJulia | I feel like we had a problem a long time ago with the mocked contents | 18:16 |
| *** gkadam has quit IRC | 18:17 | |
| TheJulia | got it | 18:20 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: fix uname mock arguments for get-pip.py https://review.openstack.org/585503 | 18:20 |
| TheJulia | I think | 18:20 |
| *** cdearborn has quit IRC | 18:28 | |
| mjturek | TheJulia: yeah I think you got it, ./fakeuname -rs exit's with a 1 | 18:28 |
| mjturek | but with yours it should succeed | 18:28 |
| TheJulia | now we get to wait... and I should make some coffee or something | 18:29 |
| * mjturek has thermos with perpetual coffee :) | 18:29 | |
| TheJulia | That sounds like quite the enchanted item | 18:33 |
| *** hamzy_ has joined #openstack-ironic | 18:39 | |
| mjturek | yeaaaah I'm exagerating. 20oz just feels infinite | 18:39 |
| openstackgerrit | Julia Kreger proposed openstack/networking-baremetal master: Update neutron-lib requirement for rocky https://review.openstack.org/585510 | 18:44 |
| TheJulia | I feel like there is a coffee rpg to be had | 18:44 |
| openstackgerrit | Julia Kreger proposed openstack/networking-generic-switch master: Update neutron-lib requirement for Rocky https://review.openstack.org/585511 | 18:46 |
| openstackgerrit | Merged openstack/ironic-tempest-plugin master: Correct the usage on listing neutron ports https://review.openstack.org/583990 | 18:52 |
| *** mjturek has quit IRC | 18:53 | |
| *** mgoddard has quit IRC | 18:56 | |
| *** alex_xu has quit IRC | 19:03 | |
| *** alex_xu has joined #openstack-ironic | 19:07 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-specs master: Follow-up ramdisk specification https://review.openstack.org/585518 | 19:13 |
| *** alexpilo_ has quit IRC | 19:18 | |
| *** ptx0 has quit IRC | 19:18 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-specs master: Cleanup specs for Rocky https://review.openstack.org/584608 | 19:20 |
| *** rwsu has quit IRC | 19:34 | |
| *** ptx0 has joined #openstack-ironic | 19:34 | |
| *** alexpilotti has joined #openstack-ironic | 19:35 | |
| TheJulia | so looks like the tempest change stuff just killed our CI jobs. Rechecked a few things... I guess we will know more in few hours | 19:39 |
| *** jcoufal has quit IRC | 19:58 | |
| *** mjturek has joined #openstack-ironic | 20:08 | |
| *** rcernin_ has joined #openstack-ironic | 20:18 | |
| *** jtomasek has quit IRC | 20:20 | |
| jroll | TheJulia: ouch. I'm basically done for the week, if you or someone else doesn't mind babysitting the conductor group things that'd be awesome | 20:23 |
| jroll | I guess I can probably recheck later tonight or in the morning | 20:23 |
| TheJulia | jroll: was already planning on it | 20:23 |
| jroll | <3 | 20:23 |
| TheJulia | jroll: thanks! have a wonderful extra long weekend :) | 20:23 |
| jroll | and you have a good week :) | 20:24 |
| *** racedo has joined #openstack-ironic | 20:30 | |
| *** baha has quit IRC | 20:32 | |
| *** d0ugal has quit IRC | 20:44 | |
| *** baha has joined #openstack-ironic | 20:46 | |
| *** mjturek has quit IRC | 20:49 | |
| *** mjturek has joined #openstack-ironic | 20:52 | |
| *** hamzy_ has quit IRC | 20:55 | |
| *** mjturek has quit IRC | 20:57 | |
| *** trown is now known as trown|outtypewww | 21:02 | |
| *** mjturek has joined #openstack-ironic | 21:06 | |
| *** mjturek has quit IRC | 21:09 | |
| *** mjturek has joined #openstack-ironic | 21:12 | |
| *** alexpilotti has quit IRC | 21:15 | |
| *** racedo has quit IRC | 21:21 | |
| *** alexpilotti has joined #openstack-ironic | 21:27 | |
| *** rcernin_ has quit IRC | 21:30 | |
| *** patchbot has quit IRC | 21:32 | |
| *** patchbot has joined #openstack-ironic | 21:33 | |
| *** patchbot has quit IRC | 21:35 | |
| *** patchbot has joined #openstack-ironic | 21:35 | |
| openstackgerrit | Olivier Bourdon proposed openstack/ironic-python-agent master: Add configurable value for get-pip.py retrieval https://review.openstack.org/585337 | 21:37 |
| TheJulia | olivierb-: https://review.openstack.org/#/c/585503/ may help :( | 21:39 |
| patchbot | patch 585503 - ironic-python-agent - fix uname mock arguments for get-pip.py | 21:39 |
| olivierb- | yes I am currentlly looking at it | 21:40 |
| olivierb- | and from zuul it seems very much OK | 21:40 |
| olivierb- | thanks | 21:40 |
| olivierb- | going to have a good night now, bye TheJulia | 21:41 |
| olivierb- | and have a nice day all ironicers | 21:42 |
| TheJulia | goodnight olivierb- | 21:43 |
| *** priteau has quit IRC | 21:51 | |
| *** dprince has quit IRC | 22:02 | |
| *** alexpilotti has quit IRC | 22:21 | |
| *** rcernin has joined #openstack-ironic | 22:30 | |
| *** hongbin has quit IRC | 22:30 | |
| *** alexpilotti has joined #openstack-ironic | 22:35 | |
| *** baha has quit IRC | 22:39 | |
| *** rh-jelabarre has quit IRC | 23:07 | |
| *** markguz has quit IRC | 23:10 | |
| *** rh-jelabarre has joined #openstack-ironic | 23:13 | |
| *** pmannidi has joined #openstack-ironic | 23:14 | |
| TheJulia | rloo: if your still around, https://review.openstack.org/#/c/585503/ could use a quick review to hopefully help sort things on the IPA gate | 23:22 |
| patchbot | patch 585503 - ironic-python-agent - fix uname mock arguments for get-pip.py | 23:22 |
| *** r-daneel has quit IRC | 23:38 | |
| *** rpioso is now known as rpioso|afk | 23:51 | |
| *** mjturek has quit IRC | 23:54 | |
| *** yolanda_ has joined #openstack-ironic | 23:56 | |
| *** linkmark has quit IRC | 23:57 | |
| *** yolanda has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
