*** bfournie has joined #openstack-ironic | 00:00 | |
*** ociuhandu has quit IRC | 00:01 | |
rpioso | TheJulia: I issued the command "openstack server create --image overcloud-full --flavor baremetal --key-name default test-server" back then. | 00:02 |
---|---|---|
TheJulia | ... | 00:02 |
TheJulia | rpioso: okay,, I'm going to go open a bottle of red wine, and try an experiment | 00:03 |
rpioso | TheJulia: Also issued this one -- "openstack baremetal node set {id} --property capabilities=boot_option:local,cpu_vt:true" | 00:03 |
rpioso | Note the boot_option:local | 00:03 |
rpioso | TheJulia: Sounds good. Thank you! | 00:03 |
*** rcernin_ has joined #openstack-ironic | 00:04 | |
*** rcernin has quit IRC | 00:07 | |
*** ociuhandu has joined #openstack-ironic | 00:10 | |
*** ociuhandu has quit IRC | 00:14 | |
*** ociuhandu has joined #openstack-ironic | 00:17 | |
TheJulia | so it is a nested partition table | 00:18 |
TheJulia | https://www.irccloud.com/pastebin/1JzBjEta/ | 00:19 |
*** rcernin_ has quit IRC | 00:19 | |
TheJulia | err hmm | 00:21 |
*** ociuhandu has quit IRC | 00:21 | |
*** ociuhandu has joined #openstack-ironic | 00:23 | |
TheJulia | so not quite, the tools are interpretting it that way | 00:23 |
*** gyee has quit IRC | 00:24 | |
TheJulia | rpioso: so overcloud-full is a partition image, I bet you had it uploaded as a partition image and ironic did the needful to make it bootable. Would you give that a try please? | 00:24 |
rpioso | TheJulia: We're thinking alike. I've been prepping my devstack environ for another go, and it's ready. | 00:25 |
TheJulia | and maybe to try a whole disk image, use diskimage-builder to make a uefi bootable image | 00:25 |
rpioso | ls | 00:25 |
TheJulia | antonym was doing similar earlier it seemed | 00:25 |
* rpioso shakes his head when he enters shell commands in IRC | 00:25 | |
*** rcernin has joined #openstack-ironic | 00:27 | |
*** ociuhandu has quit IRC | 00:28 | |
*** Goneri has joined #openstack-ironic | 00:29 | |
TheJulia | it happens | 00:30 |
rpioso | TheJulia: It's stackin' | 00:31 |
*** ociuhandu has joined #openstack-ironic | 00:31 | |
* rpioso noticed a new project has been proposed -- go-redfish | 00:33 | |
*** sshnaidm is now known as sshnaidm|afk | 00:35 | |
*** ociuhandu has quit IRC | 00:36 | |
*** ociuhandu has joined #openstack-ironic | 00:41 | |
*** ociuhandu has quit IRC | 00:46 | |
rpioso | TheJulia: The stack failed -- "Do not know what to do with overcloud-full.tar". | 00:51 |
*** ociuhandu has joined #openstack-ironic | 00:51 | |
rpioso | TheJulia: Yikes! I'll fix it! | 00:53 |
*** cdearborn has quit IRC | 00:54 | |
*** Goneri has quit IRC | 00:55 | |
*** ricolin has joined #openstack-ironic | 00:56 | |
*** ociuhandu has quit IRC | 00:57 | |
openstackgerrit | Merged openstack/ironic-python-agent-builder master: Revert "Update tinycore from 9.x to 10.x" https://review.opendev.org/698098 | 01:01 |
*** ociuhandu has joined #openstack-ironic | 01:02 | |
TheJulia | second ci fix rechecked | 01:03 |
rpioso | TheJulia: I gzipped the beast in /opt/stack/devstack/files and updated IMAGES_URL. It's stackin' again. | 01:04 |
*** ociuhandu has quit IRC | 01:06 | |
*** goldyfruit_ has joined #openstack-ironic | 01:09 | |
TheJulia | have fun | 01:10 |
* TheJulia hopes CI will be good before she falls asleep | 01:11 | |
*** ociuhandu has joined #openstack-ironic | 01:12 | |
*** ociuhandu has quit IRC | 01:16 | |
rpioso | TheJulia: Something must be wrong with IMAGES_URLS. | 01:19 |
*** ociuhandu has joined #openstack-ironic | 01:21 | |
rpioso | + functions:upload_image:393 : openstack --os-cloud=devstack-admin --os-region-name=RegionOne image create overcloud-full --property hw_rng_model=virtio --public --container-format ami --disk-format ami | 01:22 |
rpioso | /opt/stack/devstack/functions: line 393: : No such file or directory | 01:22 |
*** ociuhandu has quit IRC | 01:26 | |
*** ociuhandu has joined #openstack-ironic | 01:29 | |
rpioso | TheJulia: In what form should I stage that tarball for stack.sh's consumption, and what are the related settings in local.conf? | 01:32 |
*** ociuhandu has quit IRC | 01:37 | |
*** ociuhandu has joined #openstack-ironic | 01:42 | |
*** goldyfruit_ has quit IRC | 01:43 | |
*** ociuhandu has quit IRC | 01:47 | |
*** ociuhandu has joined #openstack-ironic | 01:49 | |
*** rloo has quit IRC | 01:53 | |
*** ociuhandu has quit IRC | 01:57 | |
*** ociuhandu has joined #openstack-ironic | 02:08 | |
*** ociuhandu has quit IRC | 02:13 | |
*** ociuhandu has joined #openstack-ironic | 02:14 | |
*** bobmel has joined #openstack-ironic | 02:20 | |
*** ociuhandu has quit IRC | 02:22 | |
*** igordc has quit IRC | 02:24 | |
*** bobmel has quit IRC | 02:25 | |
*** ociuhandu has joined #openstack-ironic | 02:38 | |
*** dmellado has quit IRC | 02:45 | |
*** vesper11 has quit IRC | 02:45 | |
*** tridde has joined #openstack-ironic | 02:45 | |
*** trident has quit IRC | 02:46 | |
*** vesper11 has joined #openstack-ironic | 02:46 | |
*** ociuhandu has quit IRC | 02:47 | |
*** dmellado has joined #openstack-ironic | 02:48 | |
rpioso | TheJulia: I made some progress. I extracted the contents of the overcloud-full tarball, renamed a couple files to match devstack's image naming convention -- "-vmlinuz" and "-initrd", with a dash -- converted the qcow2 to raw .img, and created a gzipped tarball. | 03:07 |
rpioso | TheJulia: It now "Installing the bootloader for node 761dd238-3fb3-4168-bde5-a12e69cd891b on partition a192515b-f5e5-4763-ae76-4bf20db82743, EFI system partition ..." | 03:08 |
rpioso | TheJulia: However, that fails -- "ironic.common.exception.InstanceDeployFailure: Failed to install a bootloader when deploying node 761dd238-3fb3-4168-bde5-a12e69cd891b. Error: {'message': "Command execution failed: Installing GRUB2 boot loader to device /dev/sda failed with Unexpected error while running command.\nCommand: mount /dev/sda2 /tmp/tmpHwiv0H\nExit code: 32\nStdout: u''\nStderr: u'mount: wrong fs | 03:11 |
rpioso | type, bad option, bad superblock on /dev/sda2,\\n missing codepage or helper program, or other error\\n\\n" | 03:11 |
rpioso | TheJulia: Do you have a suggestion on how to address that one? | 03:11 |
*** michchap has joined #openstack-ironic | 03:16 | |
*** ociuhandu has joined #openstack-ironic | 03:47 | |
*** ociuhandu has quit IRC | 03:51 | |
*** rh-jelabarre has quit IRC | 04:08 | |
*** mkrai has joined #openstack-ironic | 04:23 | |
*** ociuhandu has joined #openstack-ironic | 04:41 | |
*** ociuhandu has quit IRC | 04:46 | |
openstackgerrit | Steve Baker proposed openstack/metalsmith master: Implement a instance_config.NovajoinConfig https://review.opendev.org/696993 | 04:49 |
*** tzumainn has quit IRC | 04:53 | |
*** ociuhandu has joined #openstack-ironic | 05:35 | |
*** ociuhandu has quit IRC | 05:40 | |
kaifeng | I have created a story for the ipxe discussion: https://storyboard.openstack.org/#!/story/2007003 and put some context in. | 06:13 |
kaifeng | TheJulia dtantsur|afk ^^ | 06:14 |
kaifeng | just let me know if you have any concern. | 06:15 |
*** bobmel has joined #openstack-ironic | 06:22 | |
openstackgerrit | Kaifeng Wang proposed openstack/ironic master: Explicitly use ipxe as boot interface for iPXE testing https://review.opendev.org/698146 | 06:25 |
*** bobmel has quit IRC | 06:26 | |
*** ociuhandu has joined #openstack-ironic | 06:29 | |
*** jtomasek has joined #openstack-ironic | 06:31 | |
*** ociuhandu has quit IRC | 06:34 | |
openstackgerrit | Kaifeng Wang proposed openstack/ironic master: Fix ipxe interface to perform ipxe boot without ipxe_enabled enabled https://review.opendev.org/697634 | 06:55 |
*** ociuhandu has joined #openstack-ironic | 07:02 | |
*** ociuhandu has quit IRC | 07:06 | |
arne_wiebalck | Good morning kaifeng and ironic! | 07:26 |
*** ociuhandu has joined #openstack-ironic | 07:26 | |
*** ociuhandu has quit IRC | 07:32 | |
*** bobmel has joined #openstack-ironic | 07:33 | |
*** mkrai has quit IRC | 07:35 | |
kaifeng | good morning, arne_wiebalck o/ | 07:36 |
*** mkrai has joined #openstack-ironic | 07:38 | |
*** pcaruana has joined #openstack-ironic | 07:39 | |
*** ociuhandu has joined #openstack-ironic | 07:46 | |
*** rcernin has quit IRC | 07:54 | |
mkrai | good morning ironic arne_wiebalck o/ | 08:23 |
mkrai | kaifeng, gm | 08:23 |
*** tesseract has joined #openstack-ironic | 08:27 | |
*** khansa has joined #openstack-ironic | 08:28 | |
kaifeng | good morning mkrai o/ | 08:35 |
arne_wiebalck | hey mkrai o/ | 08:37 |
*** gudrutis2 has joined #openstack-ironic | 08:38 | |
*** rpittau|afk is now known as rpittau | 08:41 | |
rpittau | good morning ironic! o/ | 08:41 |
kaifeng | hey rpittau o/ | 08:42 |
rpittau | hey kaifeng :) | 08:42 |
kaifeng | rpittau: do you know what does if [ -h $something ] mean? | 08:43 |
rpittau | kaifeng: that should check if the file is a symlink | 08:44 |
rpittau | $something ^ | 08:44 |
kaifeng | rpittau: ack, thanks! | 08:44 |
*** mkrai has quit IRC | 08:45 | |
*** mkrai has joined #openstack-ironic | 08:46 | |
*** iurygregory has joined #openstack-ironic | 08:46 | |
iurygregory | good morning Ironic o/ | 08:46 |
kaifeng | hey iurygregory o/ | 08:47 |
iurygregory | kaifeng, o/ | 08:47 |
*** ociuhandu has quit IRC | 08:48 | |
kaifeng | it seems ironic-tempest-ipa-{wholedisk,partition}-bios-agent_ipmitool-tinyipa* are broken due to some ssh authentication issue in paramiko. | 08:52 |
iurygregory | kaifeng, what patch? there were some problems in ipa yesterday now things may be better... | 08:52 |
iurygregory | Julia pushed https://review.opendev.org/#/c/698098/ | 08:53 |
patchbot | patch 698098 - ironic-python-agent-builder - Revert "Update tinycore from 9.x to 10.x" (MERGED) - 1 patch set | 08:53 |
rpittau | hey iurygregory :) | 08:54 |
iurygregory | rpittau, o/ | 08:54 |
kaifeng | iurygregory: i have seen this in my patches and it seems there in the last week. | 08:54 |
mkrai | good morning iurygregory | 08:54 |
mkrai | good morning rpittau | 08:54 |
iurygregory | kaifeng, ouch =( | 08:55 |
kaifeng | iurygregory: doesn't look like related to my change.. | 08:55 |
iurygregory | mkrai, o/ | 08:55 |
mkrai | I am trying to build an IPA image but it fails with "/tmp/in_target.d/install.d/60-ironic-agent-install: line 13: /usr/share/ironic-python-agent/imagebuild/common/generate_upper_constraints.sh: No such file or directory" | 08:55 |
iurygregory | kaifeng, do you have the links? | 08:55 |
kaifeng | the 698146 | 08:56 |
mkrai | Do I need to install anything before building the image? | 08:56 |
kaifeng | mkrai: maybe you need a new version? I have seen such report recently. | 08:57 |
mkrai | kaifeng, new version of what? | 08:57 |
mkrai | kaifeng, my setup is running stein | 08:57 |
kaifeng | mkrai: https://storyboard.openstack.org/#!/2006790 | 08:58 |
*** sshnaidm|afk is now known as sshnaidm | 08:58 | |
iurygregory | kaifeng, in 698146 i don't see any logs to look .-. | 08:58 |
iurygregory | nvm | 08:59 |
iurygregory | forgot I can use zuul XD | 08:59 |
kaifeng | iurygregory: oh, its the in the top of zuul.openstack.org :) | 08:59 |
iurygregory | yeah =) | 08:59 |
*** priteau has joined #openstack-ironic | 09:00 | |
kaifeng | mkrai: i have successfully built dib image with the latest ipa-b a few weeks before, just couldn't remember which version i was using | 09:01 |
mkrai | kaifeng, /usr/share/ironic-python-agent/ i don;t have this folder | 09:03 |
mkrai | kaifeng, I installed ipa from source | 09:03 |
iurygregory | kaifeng, maybe this was the problem with the rescue tests that dtantsur|afk was talking about | 09:04 |
iurygregory | not sure if the fix is in place... | 09:04 |
kaifeng | iurygregory: is it https://review.opendev.org/698011 ? | 09:06 |
patchbot | patch 698011 - ironic-tempest-plugin - Actually test rescue in the standalone job - 1 patch set | 09:06 |
kaifeng | that sounds like we didn't really test rescue in standalone before | 09:06 |
iurygregory | yeah... | 09:06 |
kaifeng | if we have issue in the rescue test, then this patch would beat ci harder :) | 09:07 |
kaifeng | mkrai: have you tried pip3 install --user ironic-python-agent-builder and goes from there? | 09:08 |
mkrai | kaifeng, no, i used dib directly | 09:11 |
kaifeng | mkrai: i think you need to use a lower dib version since ironic-agent has been removed in the train cycle. | 09:12 |
kaifeng | and with an older ipa source | 09:13 |
mkrai | kaifeng, ok thanks i will try that | 09:14 |
*** lucasagomes has joined #openstack-ironic | 09:15 | |
*** ociuhandu has joined #openstack-ironic | 09:17 | |
openstackgerrit | Pradip Kadam proposed openstack/ironic master: DRAC : idrac-redfish inspect updates pxe port https://review.opendev.org/693328 | 09:20 |
*** ociuhandu has quit IRC | 09:22 | |
openstackgerrit | Rachit Kapadia proposed openstack/ironic master: DRAC: Drives conversion from raid to jbod https://review.opendev.org/671038 | 09:24 |
*** dougsz has joined #openstack-ironic | 09:30 | |
kaifeng | iurygregory: i suspect you are right, the broken jobs seem related with rescue tests.. | 09:31 |
openstackgerrit | Merged openstack/ironic-python-agent-builder master: Update tinycore mirrors https://review.opendev.org/698047 | 09:31 |
iurygregory | this would be good XD since at least we know | 09:31 |
kaifeng | feel like tinycore has some issue on providing ssh service after bumping | 09:31 |
kaifeng | iurygregory: agreed :) | 09:33 |
*** alexmcleod has joined #openstack-ironic | 09:34 | |
rpittau | kaifeng: after moving back to 9.x (merged already) it should be ok, I see all the jobs that were failing are passing now | 09:36 |
*** derekh has joined #openstack-ironic | 09:37 | |
*** ociuhandu has joined #openstack-ironic | 09:42 | |
openstackgerrit | Rachit Kapadia proposed openstack/ironic master: DRAC: Drives conversion from raid to jbod https://review.opendev.org/671038 | 10:02 |
*** priteau has quit IRC | 10:05 | |
*** dtantsur|afk is now known as dtantsur | 10:15 | |
*** derekh has quit IRC | 10:20 | |
dtantsur | morning ironic | 10:26 |
*** etingof has quit IRC | 10:26 | |
openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: CI: disable IPv6 in neutron https://review.opendev.org/698012 | 10:26 |
iurygregory | morning dtantsur | 10:26 |
dtantsur | arne_wiebalck, rpittau, kaifeng could you check ^^ asap | 10:26 |
openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: CI: re-enable testing rescue https://review.opendev.org/698207 | 10:27 |
rpittau | dtantsur: done, I skipped that, so much chaos today | 10:27 |
*** etingof has joined #openstack-ironic | 10:29 | |
arne_wiebalck | dtantsur: done | 10:30 |
openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: CI: re-enable testing rescue https://review.opendev.org/698207 | 10:32 |
kaifeng | uh, late for the party, i just can resist food :) | 10:43 |
rpittau | :D | 10:44 |
dtantsur | no need to resist food! | 10:44 |
iurygregory | food is always good =D | 10:45 |
kaifeng | especially in a state of hungry :) | 10:46 |
kaifeng | rpittau: the revert patch (698098) merged before my propose a patch, but I still see the jobs failing in 698146 | 10:51 |
dtantsur | I think we have a lot of overlapping problems | 10:52 |
dtantsur | Rescue hard-broken is only one of them | 10:52 |
rpittau | kaifeng: that's a different issue, yeah :/ | 10:52 |
* dtantsur is impressed that some patches actually do merge | 10:53 | |
iurygregory | dtantsur, it's magic! | 10:54 |
*** dsneddon has quit IRC | 11:08 | |
openstackgerrit | Kaifeng Wang proposed openstack/ironic master: Remove the [pxe]ipxe_enabled configuration option https://review.opendev.org/696810 | 11:11 |
*** ociuhandu has quit IRC | 11:13 | |
*** ociuhandu has joined #openstack-ironic | 11:13 | |
openstackgerrit | Kaifeng Wang proposed openstack/ironic master: Start removing ipxe support from the pxe interface https://review.opendev.org/696666 | 11:13 |
*** dsneddon has joined #openstack-ironic | 11:17 | |
mkrai | kaifeng, I used ironic-python-agent-builder still the same issue exist. http://paste.openstack.org/show/787361/ | 11:19 |
dtantsur | mkrai: what's the command line you're using? | 11:19 |
mkrai | dtantsur, ironic-python-agent-builder -o my-ipa -e devuser debian | 11:19 |
dtantsur | mkrai: which version of IPA-builder do you have? make sure you have the latest | 11:20 |
*** bobmel has quit IRC | 11:20 | |
mkrai | dtantsur, 1.0.0 | 11:21 |
*** bobmel has joined #openstack-ironic | 11:21 | |
mkrai | dtantsur, 1.10 is the latest? https://tarballs.openstack.org/ironic-python-agent-builder/ | 11:21 |
mkrai | * 1.1.0 | 11:22 |
dtantsur | mkrai: grab 1.1.0, I think it'll fix your issue | 11:22 |
mkrai | Ok thanks dtantsur | 11:22 |
*** dsneddon has quit IRC | 11:23 | |
kaifeng | checked my building env, i was using ipa-b 1.1.0 | 11:24 |
*** ociuhandu has quit IRC | 11:28 | |
mkrai | dtantsur, thanks it worked | 11:32 |
dtantsur | I'm glad to hear it | 11:33 |
kaifeng | dtantsur: do you think it's a duplicate thing if we put some useful variables and elements in the ipa-b documents? | 11:36 |
dtantsur | kaifeng: you mean, duplicating with DIB? | 11:36 |
dtantsur | I think it's fine as long as they're really relevant to our case | 11:37 |
dtantsur | likely to be used | 11:37 |
kaifeng | dtantsur: i am identifying an issue that the network name can't be renamed to the name of consistent naming | 11:37 |
kaifeng | i am seeing ipa collected eth0 instead of enN or ensN or such thing, and at last identified there is an element stable-interface-names helps with this | 11:39 |
kaifeng | dtantsur: i don't want to duplicate docs with DIB, but if we have some pointers there, that may help users building their images. before today, i even don't know this element exists. | 11:40 |
dtantsur | me neither | 11:40 |
dtantsur | yeah, this one seems potentially relevant | 11:40 |
*** ijw has joined #openstack-ironic | 11:43 | |
kaifeng | dtantsur: okay, i may collect something for a while and then put an update to the ipa-b docs. | 11:43 |
*** ijw has quit IRC | 11:47 | |
dtantsur | ++ thx | 11:52 |
* dtantsur IKEA time, back later | 11:53 | |
*** dtantsur is now known as dtantsur|ikea | 11:54 | |
mkrai | kaifeng, I created the custom ipa image with devsuser, set it to cleaning and it is stuck in clean-wait | 11:54 |
mkrai | kaifeng, How can I check if IPA is running fine or not? | 11:54 |
mkrai | kaifeng, this document https://docs.openstack.org/ironic-python-agent/latest/admin/troubleshooting.html#diskimage-builder-dib only tells how to create images for debugging purpose | 11:55 |
kaifeng | mkrai: you are using real machine right? | 11:55 |
mkrai | kaifeng, yes | 11:56 |
mkrai | I can see the console | 11:56 |
mkrai | It prompts "debian login" | 11:56 |
kaifeng | mkrai: usually i use kvm to see what happens if something goes wrong | 11:56 |
mkrai | kaifeng, I saw PXE and tftp happened fine | 11:56 |
kaifeng | you can use the user to login into the ramdisk | 11:57 |
mkrai | kaifeng, it prompts for password :( | 11:57 |
kaifeng | the value you set to DIB_DEV_USER_USERNAME/PASSWORD before image building | 11:57 |
mkrai | kaifeng, Ohh I didn't set password | 11:57 |
kaifeng | oh sure, you'll need that, password or publickey | 11:58 |
*** dsneddon has joined #openstack-ironic | 11:58 | |
kaifeng | you can refer to the dib documentation for the environmental variable required | 11:58 |
kaifeng | should be under the devuser element | 11:58 |
mkrai | kaifeng, ok i will check that | 11:59 |
*** lucasagomes is now known as lucas-hungry | 12:02 | |
kaifeng | mkrai: since the pxe goes well, make sure you have correct callback url to ironic api from the ipa. typically this means ipa is unable to reach ironic api to continue steps followed. | 12:07 |
mkrai | kaifeng, Ok I will check that once I get access to IPA | 12:08 |
mkrai | kaifeng, If the nodes become available it means the IPA is running just fine right? | 12:09 |
kaifeng | mkrai: yeah, any state not ends with "failed" implies a successful op | 12:11 |
mkrai | kaifeng, ok so I already have such ramdisk images which move node to available. My actual issue is the node provisioning fails with | 12:13 |
mkrai | "no boot filename received" | 12:13 |
kaifeng | mkrai: it's the same issue yesterday? | 12:14 |
mkrai | kaifeng, yes :( | 12:15 |
kaifeng | mkrai: i suggest try localboot to make it work, netboot in neutron network needs more work. | 12:15 |
mkrai | kaifeng, by more work do you mean to create an OVS bridge and so | 12:16 |
mkrai | kaifeng, I tried localboot yesterday suggested by dtantsur|ikea, it also failed | 12:16 |
kaifeng | mkrai: so stuck in wait callback? | 12:17 |
kaifeng | mkrai: if you successfully provision a node to local boot, but still failed to startup, make sure the instance image is bootable from disk. | 12:19 |
mkrai | kaifeng, no it dfailed in provisioning itself | 12:19 |
mkrai | kaifeng, http://paste.openstack.org/show/787334/ | 12:20 |
kaifeng | mkrai: this is ip accessing instance image from swift failed | 12:21 |
kaifeng | s/ip/ipa/ | 12:22 |
*** ociuhandu has joined #openstack-ironic | 12:23 | |
kaifeng | mkrai: sorry i got to go now, i suggest you use iscsi which has lesser requirements on the environment. | 12:23 |
mkrai | kaifeng, ok thanks for your help | 12:24 |
mkrai | kaifeng, bye! | 12:24 |
kaifeng | mkrai: if you are sticking to agent driver, make sure ipa has access to the glance api, and tempurl thing is set correctly. | 12:24 |
*** kaifeng is now known as kaifeng|afk | 12:24 | |
*** ociuhandu has quit IRC | 12:27 | |
*** bobmel has quit IRC | 12:29 | |
*** bobmel has joined #openstack-ironic | 12:30 | |
*** mkrai has quit IRC | 12:48 | |
*** bfournie has quit IRC | 12:51 | |
*** rh-jelabarre has joined #openstack-ironic | 12:55 | |
*** lucas-hungry is now known as lucasagomes | 13:06 | |
*** rloo has joined #openstack-ironic | 13:35 | |
*** goldyfruit_ has joined #openstack-ironic | 13:37 | |
*** yaawang has quit IRC | 13:39 | |
*** yaawang has joined #openstack-ironic | 13:39 | |
*** goldyfruit___ has joined #openstack-ironic | 13:41 | |
*** bfournie has joined #openstack-ironic | 13:42 | |
etingof | TheJulia, o/ x/sushy-oem-idrac - does it need proper OS docs? I understand it should NOT have reno, but what about docs? | 13:42 |
*** Goneri has joined #openstack-ironic | 13:43 | |
*** goldyfruit_ has quit IRC | 13:44 | |
*** tkajinam has joined #openstack-ironic | 14:01 | |
*** zaneb has quit IRC | 14:09 | |
*** dsneddon has quit IRC | 14:12 | |
*** dsneddon has joined #openstack-ironic | 14:15 | |
*** dsneddon has quit IRC | 14:20 | |
TheJulia | docs could be useful, but they would be published to rtfd | 14:22 |
*** dsneddon has joined #openstack-ironic | 14:22 | |
*** mkrai has joined #openstack-ironic | 14:23 | |
iurygregory | good morning TheJulia o/ | 14:23 |
TheJulia | do we need to drop tests and shove https://review.opendev.org/#/c/698012/ in ? | 14:30 |
patchbot | patch 698012 - ironic - CI: disable IPv6 in neutron - 2 patch sets | 14:30 |
*** dsneddon has quit IRC | 14:33 | |
*** ociuhandu has joined #openstack-ironic | 14:35 | |
*** mmethot has joined #openstack-ironic | 14:35 | |
rpittau | TheJulia: I just rechecked that, it's kind of needed to fix CI | 14:35 |
*** mmethot is now known as mmethot|conferen | 14:35 | |
*** pcaruana has quit IRC | 14:36 | |
TheJulia | I understand, I'm wondering if we're going to be able to merge to master at all without ipa fix merging | 14:37 |
rpittau | TheJulia: it failed on standalone only | 14:37 |
*** tzumainn has joined #openstack-ironic | 14:39 | |
TheJulia | no, it could fail on any job that had rescue testing enabled | 14:40 |
TheJulia | of course, yes, the last attempt only failed on standalone | 14:40 |
*** tkajinam has quit IRC | 14:52 | |
*** zaneb has joined #openstack-ironic | 14:52 | |
TheJulia | it looks like the rescue explit tests in standalone just completely tanked | 14:53 |
TheJulia | never got far enough to actually test rescue :( | 14:54 |
*** igordc has joined #openstack-ironic | 14:58 | |
rpittau | yeah but I'm optymist for some obscure reason | 14:59 |
*** dsneddon has joined #openstack-ironic | 15:00 | |
TheJulia | if it doesn't pass on the first round of trying to roll throught the gate today, I'll just force it in. We can't be dead in the water for a week | 15:00 |
rpittau | wondering if it would make sense to have a recheck on single jobs in zuul, and wondering if I'm the first one to think about that | 15:00 |
rpittau | TheJulia: agreed | 15:00 |
TheJulia | not the first and there is a... umm... fundimental disagreement with the zuul team on that. | 15:00 |
TheJulia | We're alsos uper job heavy | 15:00 |
rpittau | oh interesting | 15:01 |
TheJulia | super job heavy | 15:01 |
*** pcaruana has joined #openstack-ironic | 15:01 | |
*** zaneb has quit IRC | 15:02 | |
* rpittau loves poking hornet's nests | 15:02 | |
* iurygregory likes rpittau idea | 15:03 | |
* iurygregory is not going to poke any hornet's nests | 15:04 | |
*** dsneddon has quit IRC | 15:05 | |
iurygregory | TheJulia, the sync command would be after the rmtree? | 15:05 |
* iurygregory is not sure where to put, I was thinking before the first unmount on fnally block or in the end of the try | 15:06 | |
*** ijw has joined #openstack-ironic | 15:10 | |
*** seba has joined #openstack-ironic | 15:14 | |
*** bobmel has quit IRC | 15:15 | |
*** bnemec has quit IRC | 15:15 | |
*** munimeha1 has joined #openstack-ironic | 15:18 | |
openstackgerrit | Julia Kreger proposed openstack/ironic master: Pre-shared agent token https://review.opendev.org/692609 | 15:19 |
openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent-builder master: Upgrade syslinux version https://review.opendev.org/698272 | 15:23 |
openstackgerrit | Eric Fried proposed openstack/ironic master: Update nova os-server-external-events response logic https://review.opendev.org/698273 | 15:28 |
iurygregory | rpittau, do you think in the case of we have the efi_system_part_uuid but for some reason we can't get the partition we should search for a valid partition? http://paste.openstack.org/show/787377/ | 15:28 |
openstackgerrit | Tzu-Mainn Chen proposed openstack/ironic master: Restrict ability to change owner on provisioned node https://review.opendev.org/698078 | 15:31 |
*** hjensas has quit IRC | 15:35 | |
*** ijw has quit IRC | 15:40 | |
*** dsneddon has joined #openstack-ironic | 15:40 | |
TheJulia | iurygregory: eh, after unmounts | 15:46 |
rpittau | iurygregory: mmm if the uuid doesn't correspond to a partition there may be something very wrong | 15:48 |
iurygregory | TheJulia, rpittau tks! | 15:55 |
rpittau | iurygregory: that is taken directly from the partition creation, I doubt we will end up there if anything goes wrong during disk partitioning | 15:57 |
*** goldyfruit_ has joined #openstack-ironic | 15:57 | |
*** goldyfruit___ has quit IRC | 16:00 | |
*** ociuhandu has quit IRC | 16:02 | |
*** dtantsur|ikea is now known as dtantsur | 16:02 | |
dtantsur | morning TheJulia | 16:04 |
dtantsur | so, https://review.opendev.org/#/c/698012/ doesn't seem to help, does it? >_< | 16:04 |
patchbot | patch 698012 - ironic - CI: disable IPv6 in neutron - 2 patch sets | 16:04 |
dtantsur | do we have any options left other than disabling the standalone job? | 16:05 |
dtantsur | rpittau: ^^^ | 16:05 |
iurygregory | dtantsur, it does I think | 16:06 |
iurygregory | ironic-standalone is green on zuul | 16:06 |
*** zaneb has joined #openstack-ironic | 16:06 | |
dtantsur | iurygregory: it's not on the last run | 16:06 |
dtantsur | and again we see DHCP issues https://75e2faadb9d5832df0cd-a7818f858a3bcdda33f05669617cb625.ssl.cf2.rackcdn.com/698012/2/check/ironic-standalone/172cef9/controller/logs/ironic-bm-logs/node-5_console_log.txt.gz | 16:06 |
dtantsur | just this time there are no IPv6 addresses, so no addresses at all | 16:07 |
iurygregory | D: | 16:07 |
*** ijw has joined #openstack-ironic | 16:09 | |
dtantsur | what's even more suspicious: PXE boot retries (that we've recently introduced) don't help | 16:09 |
dtantsur | so we have 3 failures to DHCP in a row | 16:09 |
dtantsur | see e.g. node-1 logs in https://f67ae6e1c6d7074fe459-ba5467d2de73878d3dd4296de3d69c75.ssl.cf2.rackcdn.com/698012/1/check/ironic-standalone/142504c/controller/logs/ironic-bm-logs/ | 16:09 |
rpittau | should we force ipv4 ? | 16:11 |
*** igordc has quit IRC | 16:11 | |
openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: CI: do not enable rescue on indirect jobs https://review.opendev.org/698207 | 16:12 |
dtantsur | rpittau: we tried in https://review.opendev.org/#/c/698012/ | 16:12 |
patchbot | patch 698012 - ironic - CI: disable IPv6 in neutron (ABANDONED) - 2 patch sets | 16:12 |
dtantsur | we end up with no addresses at all | 16:12 |
rpittau | dtantsur: sorry, yeah, still too much enthropy today | 16:12 |
openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP ipv6 jobs https://review.opendev.org/613169 | 16:14 |
*** zaneb has quit IRC | 16:15 | |
*** bobmel has joined #openstack-ironic | 16:16 | |
TheJulia | hmmmmm | 16:16 |
dtantsur | something that surprises me is that according to http://zuul.openstack.org/builds?job_name=ironic-standalone the job does pass most of the time | 16:16 |
dtantsur | while my feeling is that it fails in 90% cases | 16:17 |
dtantsur | http://zuul.openstack.org/builds?job_name=ironic-standalone-ipa-src shows a worse picture, but still not catastrophic | 16:17 |
openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-python-agent master: Avoid grub2-install when on UEFI boot mode https://review.opendev.org/696914 | 16:18 |
TheJulia | are we sure that neutron didn't bind ports or update dhcp like 10-15 seconds after we booted the VM? | 16:18 |
*** iurygregory has quit IRC | 16:18 | |
TheJulia | like 1000% sure? | 16:18 |
rpittau | TheJulia: that's what I was checking too | 16:18 |
TheJulia | because we've seen it enough that its a known thing that can happen, and why we wanted to callback stuffs | 16:18 |
*** iurygregory has joined #openstack-ironic | 16:19 | |
*** ricolin has quit IRC | 16:19 | |
dtantsur | TheJulia: I'm not sure at all | 16:20 |
dtantsur | now, what surprises me as well is http://zuul.openstack.org/builds?job_name=ironic-standalone&change=693017 | 16:20 |
dtantsur | this is etingof's CI job patch. it shows a surprising failure rate.. | 16:20 |
dtantsur | TheJulia: the neutron hypothesis would explain why the failures happen more often on our most busy job | 16:21 |
dtantsur | and it's relatively easy to test it! | 16:21 |
TheJulia | nasty neutron errors | 16:22 |
TheJulia | like... around dhcp updates | 16:22 |
etingof | that's perfectly understandable - ipmi job is trying to keep its foot in the door | 16:22 |
rpittau | I'm seeing an exception in the neutron dhcp agent logs, not sure it's related | 16:23 |
openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: DNM raise port_setup_delay substantially https://review.opendev.org/698289 | 16:23 |
dtantsur | etingof: that grumpy old ipmi! | 16:23 |
dtantsur | TheJulia: let's test your idea ^^^ | 16:23 |
TheJulia | not going to always work, fun error incoming | 16:23 |
rpittau | Dec 10 11:09:02.674224 ubuntu-bionic-rax-ord-0013309750 neutron-dhcp-agent[20060]: ERROR neutron.agent.dhcp.agent [-] Unable to reload_allocations dhcp for 2d09d457-e7ab-490c-92a6-55518600d486.: multiprocessing.managers.RemoteError: | 16:23 |
TheJulia | https://www.irccloud.com/pastebin/LUhfa1KS/ | 16:23 |
dtantsur | fun? I like fun | 16:23 |
* TheJulia hands dtantsur whiskey | 16:23 | |
TheJulia | and points at link | 16:24 |
rpittau | there you go :) | 16:24 |
dtantsur | uuuuuuuuuuuugh | 16:24 |
dtantsur | who is going to #openstack-neutron? everyone? :) | 16:24 |
rpittau | :D | 16:24 |
iurygregory | lets join the party yay | 16:24 |
iurygregory | grab the drink that you like XD | 16:25 |
* dtantsur wants a shot of sweet nice working CI | 16:25 | |
iurygregory | ^ this is the gift for xmas... | 16:26 |
rpioso | Good morning, ironic | 16:26 |
rpittau | hey rpioso :) | 16:27 |
iurygregory | morning rpioso o/ | 16:27 |
*** hjensas has joined #openstack-ironic | 16:29 | |
rpioso | dtantsur: I made more progress getting a devstack local install. I used the RDO overcloud-full tarball and modified it to make stack.sh at least somewhat happy with it. The deployment failed toward the very end -- "ironic.common.exception.InstanceDeployFailure: Failed to install a bootloader when deploying node 761dd238-3fb3-4168-bde5-a12e69cd891b. Error: {'message': "Command execution failed: Installing GRUB2 | 16:29 |
rpioso | boot loader to device /dev/sda failed with Unexpected error while running command.\nCommand: mount /dev/sda2 /tmp/tmpHwiv0H\nExit code: 32\nStdout: u''\nStderr: u'mount: wrong fs type, bad option, bad superblock on /dev/sda2,\\n missing codepage or helper program, or other error\\n\\n" | 16:29 |
rpioso | dtantsur: Any idea how to address that one? | 16:29 |
rpioso | iurygregory, rpittau: \o | 16:29 |
dtantsur | rpioso: overcloud-full is a partition image, did you add kernel/ramdisk to glance? | 16:29 |
*** zaneb has joined #openstack-ironic | 16:30 | |
rpioso | dtantsur: Yes, I believe they're there. Three elements were uploaded. | 16:30 |
dtantsur | rpioso: and linked from the main image? | 16:31 |
rpioso | dtantsur: How can I tell? | 16:31 |
rpioso | dtantsur: The main image has a ramdisk_id property which contains the ID of the ramdisk image. | 16:32 |
dtantsur | rpioso: compare http://paste.openstack.org/show/787384/ | 16:33 |
rpioso | dtanstur: Same with the kernel_id. | 16:33 |
dtantsur | ah, good | 16:33 |
dtantsur | worth double-checking that the right image was uploaded as the root one | 16:35 |
rpioso | dtantsur: I don't have a cirros-0.4.0-x86_64-disk like thing, though. Just three items from overcloud-full are uploaded. | 16:35 |
dtantsur | rpioso: yeah, I should you it for an example, I don't have an environment with overcloud-full at hand | 16:36 |
*** gyee has joined #openstack-ironic | 16:37 | |
rpioso | dtantsur: http://paste.openstack.org/show/787385/ | 16:39 |
rpioso | dtantsur: What are cirros-d160722-x86_64-disk and cirros-d160722-x86_64-uec ? | 16:40 |
dtantsur | rpioso: cirros images, used for testing | 16:40 |
dtantsur | disk is a whole disk image, uec - partition | 16:40 |
rpioso | dtantsur: I see. | 16:41 |
*** goldyfruit___ has joined #openstack-ironic | 16:41 | |
rpioso | dtantsur: Do you know what /dev/sda2 is expected to contain? | 16:43 |
dtantsur | rpioso: I assume it's your root partition | 16:43 |
*** goldyfruit_ has quit IRC | 16:44 | |
rpioso | dtantsur: Might a lack of root device hints be causing that error? | 16:45 |
rpioso | My impression is that the root partition was successfully written to the disk before the mount. Correct? | 16:45 |
TheJulia | is raw streaming turned on? | 16:46 |
dtantsur | rpioso: it was, that's why I suspect the contents of the image. | 16:46 |
rpioso | TheJulia: Please elaborate. I'm using iSCSI deploy. | 16:47 |
dtantsur | root hints only define the device itself, they have nothing to do with partitions | 16:47 |
dtantsur | if iSCSI, then raw streaming is off | 16:47 |
*** mmethot|conferen is now known as mmethot | 16:48 | |
TheJulia | I'd second image contents, I'd try to make sure that it was the qcow that was upload and that the qcow can be read/mounted locally | 16:48 |
rpioso | Oh! I have some further insight. I used qemu-img to convert the qcow2 to raw in order to placate devstack's deployment. At least I believe I was placating it. | 16:50 |
rpioso | I had to rename the kernel and ramdisk partitions and create a new zipped tarball. | 16:50 |
rpioso | It appears functions.update_image expects a dash in the ramdisk and kernel file names. Should I just rename the qcow2 by removing its extension? | 16:52 |
dtantsur | rpioso: it should be fine to just use qcow2. which problems did you have with it? | 16:52 |
rpioso | dtantsur: The file names. | 16:52 |
dtantsur | rpioso: I'm quite confident the file names do not affect ironic | 16:52 |
rpioso | dtantsur: They affect the devstack stack.sh. | 16:53 |
dtantsur | but renaming files don't require converting between the formats, so I'm still a bit confused | 16:53 |
rpioso | I made an assumption that no extension implied a desire for raw. | 16:53 |
dtantsur | rpioso: you can just upload the image yourself, it's not very tricky. I think we have the docs somewhere. | 16:53 |
*** ijw_ has joined #openstack-ironic | 16:53 | |
TheJulia | how about a diff of what you did to the devstack plugin because there is code to loop through stuff and do things, and it sounds like you might bave run afoul of that code | 16:53 |
TheJulia | and then gotten bad content uploaded or uploaded with the wrong flags | 16:54 |
dtantsur | check the user image part in https://docs.openstack.org/ironic/latest/install/configure-glance-images.html | 16:54 |
rpioso | https://opendev.org/openstack/devstack/src/commit/ede8b1269cb03178c31f700200309dd76b00dd5a/functions#L297-L315 | 16:55 |
rpioso | dtantsur: I'll follow those instructions and give it another go. | 16:56 |
rpioso | dtantsur, TheJulia: Thank you! | 16:56 |
*** ijw has quit IRC | 16:58 | |
*** tesseract has quit IRC | 16:58 | |
*** mkrai has quit IRC | 17:02 | |
*** iurygregory has quit IRC | 17:02 | |
*** lucasagomes has quit IRC | 17:04 | |
*** bobmel has quit IRC | 17:11 | |
* rpioso grins at the penguins displayed during the successful local boot | 17:15 | |
rpioso | dtantsur, TheJulia: Success! \o/ | 17:15 |
dtantsur | great! | 17:16 |
rpioso | Many thanks! | 17:17 |
TheJulia | woot | 17:18 |
*** hjensas has quit IRC | 17:20 | |
rpioso | Do you know the overcloud-full CentOS 7 root or other creds? | 17:23 |
rpioso | It's actually accepting my keystrokes. | 17:23 |
TheJulia | root and the key you supplied for the deploy? | 17:24 |
rpioso | SSH? | 17:24 |
*** ociuhandu has joined #openstack-ironic | 17:25 | |
TheJulia | yes | 17:25 |
*** ociuhandu has quit IRC | 17:29 | |
rpittau | good night! o/ | 17:29 |
*** rpittau is now known as rpittau|afk | 17:29 | |
rpioso | rpittau|afk: Nite :-) | 17:32 |
* TheJulia wants to go back to sleep | 17:35 | |
TheJulia | daystar move faster! | 17:35 |
TheJulia | dtantsur: on a plus side https://review.opendev.org/#/c/692609/ has a +1 from zuul | 17:37 |
patchbot | patch 692609 - ironic - Pre-shared agent token - 7 patch sets | 17:37 |
dtantsur | at least something has a +1 from zuul | 17:37 |
* dtantsur puts on his queue | 17:37 | |
TheJulia | \o/ | 17:38 |
dtantsur | rpioso: no default creds. you need to use ssh keypairs (if using with nova) | 17:38 |
dtantsur | time to go now, g'night | 17:38 |
*** dtantsur is now known as dtantsur|afk | 17:38 | |
rpioso | dtantsur|afk: Thanks. No nova, though. | 17:38 |
dtantsur|afk | rpioso: try metalsmith :) | 17:39 |
dtantsur|afk | (disclaimer: I wrote it) | 17:39 |
dtantsur|afk | it's a CLI for ironic without nova, can handle things like networks, ssh keys, etc | 17:39 |
* dtantsur|afk goes for real now | 17:39 | |
rpioso | dtantsur|afk: I know well :-) Is it easier to deploy on real HW than devstack? | 17:39 |
*** sshnaidm is now known as sshnaidm|afk | 17:50 | |
*** khansa has quit IRC | 17:57 | |
openstackgerrit | Julia Kreger proposed openstack/ironic master: Pre-shared agent token https://review.opendev.org/692609 | 17:58 |
*** dougsz has quit IRC | 17:58 | |
openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP: Enable agent_token for virtual media boot https://review.opendev.org/697584 | 17:58 |
openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP: Send our token back to the agent https://review.opendev.org/697585 | 17:58 |
openstackgerrit | Mark Goddard proposed openstack/ironic master: WIP: Decompose core deploy step https://review.opendev.org/698309 | 17:59 |
mgoddard | arne_wiebalck, dtantsur|afk: ^ decomposed core deploy step, booted a node using the iscsi deploy interface :) | 18:00 |
*** bobmel has joined #openstack-ironic | 18:06 | |
rajinir | <TheJulia>, dtansur,rpioso and everyone , Dell Ironic CI is back online. I have updated the whiteboard | 18:06 |
*** ijw_ has quit IRC | 18:07 | |
*** ijw has joined #openstack-ironic | 18:09 | |
arne_wiebalck | mgoddard: wow, nice! | 18:09 |
rpioso | rajinir: Thank you! | 18:11 |
rpioso | iurygregory: ^^^ | 18:11 |
*** tzumainn has quit IRC | 18:12 | |
TheJulia | rajinir: thanks! one less email to send out \o/ | 18:20 |
*** goldyfruit_ has joined #openstack-ironic | 18:22 | |
*** tzumainn has joined #openstack-ironic | 18:24 | |
*** goldyfruit___ has quit IRC | 18:25 | |
*** ijw has quit IRC | 18:26 | |
*** ijw has joined #openstack-ironic | 18:27 | |
*** ijw has quit IRC | 18:27 | |
*** ijw has joined #openstack-ironic | 18:28 | |
*** bobmel has quit IRC | 18:38 | |
*** ociuhandu has joined #openstack-ironic | 18:39 | |
*** bobmel has joined #openstack-ironic | 18:41 | |
*** ociuhandu has quit IRC | 18:44 | |
*** ijw_ has joined #openstack-ironic | 18:45 | |
*** mmethot has quit IRC | 18:46 | |
*** mmethot has joined #openstack-ironic | 18:46 | |
*** bobmel has quit IRC | 18:46 | |
*** ijw has quit IRC | 18:48 | |
openstackgerrit | Tzu-Mainn Chen proposed openstack/ironic master: Restrict ability to change owner on provisioned node https://review.opendev.org/698078 | 18:56 |
openstackgerrit | Tzu-Mainn Chen proposed openstack/ironic master: Add owner to allocations and create relevant policies https://review.opendev.org/696707 | 19:00 |
openstackgerrit | Tzu-Mainn Chen proposed openstack/ironic master: Allow node owners to administer associated ports https://review.opendev.org/695255 | 19:05 |
TheJulia | "zuul ate the jobs" | 19:09 |
*** tzumainn has quit IRC | 19:12 | |
* TheJulia sighs as everything re-queues in zuul | 19:37 | |
*** igordc has joined #openstack-ironic | 19:39 | |
*** gmann is now known as gmann_afk | 19:40 | |
*** dsneddon has quit IRC | 19:58 | |
*** bobmel has joined #openstack-ironic | 20:01 | |
*** tzumainn has joined #openstack-ironic | 20:02 | |
*** bobmel has quit IRC | 20:05 | |
*** dsneddon has joined #openstack-ironic | 20:14 | |
*** rcernin has joined #openstack-ironic | 20:16 | |
*** pcaruana has quit IRC | 20:16 | |
*** dsneddon has quit IRC | 20:19 | |
*** hjensas has joined #openstack-ironic | 20:27 | |
*** Goneri has quit IRC | 20:38 | |
*** dsneddon has joined #openstack-ironic | 20:40 | |
*** dsneddon has quit IRC | 20:40 | |
*** bobmel has joined #openstack-ironic | 20:41 | |
*** naved001 has quit IRC | 20:44 | |
*** naved001 has joined #openstack-ironic | 20:44 | |
*** naved001 has quit IRC | 21:01 | |
*** naved001 has joined #openstack-ironic | 21:01 | |
*** jtomasek has quit IRC | 21:02 | |
*** goldyfruit_ has quit IRC | 21:24 | |
*** goldyfruit_ has joined #openstack-ironic | 21:30 | |
*** dsneddon has joined #openstack-ironic | 21:32 | |
*** bfournie has quit IRC | 21:57 | |
*** dhellmann has quit IRC | 22:13 | |
*** dhellmann has joined #openstack-ironic | 22:32 | |
*** bfournie has joined #openstack-ironic | 22:54 | |
*** gmann_afk is now known as gmann | 23:06 | |
openstackgerrit | Merged openstack/ironic-python-agent master: Stop the API so rescue can exit and succeed https://review.opendev.org/697844 | 23:10 |
*** rh-jelabarre has quit IRC | 23:15 | |
*** goldyfruit_ has quit IRC | 23:30 | |
*** Goneri has joined #openstack-ironic | 23:35 | |
*** Goneri has quit IRC | 23:40 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!