| *** openstackgerrit has joined #openstack-dib | 00:26 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add initramfs-tools for ubuntu-minimal https://review.openstack.org/505857 | 00:26 |
|---|---|---|
| *** openstackgerrit has quit IRC | 00:41 | |
| *** Sukhdev has quit IRC | 01:39 | |
| *** strigazi has quit IRC | 01:56 | |
| *** strigazi has joined #openstack-dib | 01:58 | |
| *** openstackgerrit has joined #openstack-dib | 02:01 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add initramfs-tools for ubuntu-minimal https://review.openstack.org/505857 | 02:01 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Avoid networkx 2.0 release https://review.openstack.org/505908 | 02:01 |
| *** yolanda has quit IRC | 02:34 | |
| *** yolanda has joined #openstack-dib | 02:35 | |
| *** Sukhdev has joined #openstack-dib | 03:24 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Support networkx 2.0 https://review.openstack.org/505963 | 04:09 |
| *** andreas-f has joined #openstack-dib | 04:42 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add initramfs-tools for ubuntu-minimal https://review.openstack.org/505857 | 04:43 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add initramfs-tools for ubuntu-minimal/pin networkx <2.0 https://review.openstack.org/505857 | 04:49 |
| ianw | prometheanfire: ^ hopefully once this all falls out, that gets the gate going again | 04:49 |
| prometheanfire | yarp | 04:49 |
| prometheanfire | tomorrow is my scheduled build the gentoo images day | 04:50 |
| ianw | i guess we need to obey upper-constraints | 04:53 |
| prometheanfire | it is sorta enforced :P | 04:55 |
| ianw | don't we need a tox_install.sh and stuff? | 04:57 |
| prometheanfire | yes, let me take a look | 04:58 |
| prometheanfire | ya, the tox file needs updating | 04:58 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder master: Use upper-constraints in tox tests https://review.openstack.org/505975 | 05:00 |
| prometheanfire | ianw: that should fix ya ^ | 05:00 |
| prometheanfire | I think that's the only place it's needed... | 05:00 |
| prometheanfire | so says the grep on the nova repo :D | 05:01 |
| prometheanfire | ianw: that will probably still need your gate fixing stuff to fix the other breakage though | 05:02 |
| prometheanfire | so feel free to update it | 05:02 |
| prometheanfire | not sure which review that was in | 05:02 |
| *** Sukhdev has quit IRC | 05:23 | |
| *** ChanServ sets mode: +o openstack | 07:18 | |
| *** hashar has joined #openstack-dib | 07:29 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Add initramfs-tools for ubuntu-minimal https://review.openstack.org/505857 | 08:08 |
| yolanda | hi, resuming the job from Ian... so trying to land fixes one by one | 08:08 |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Add initramfs-tools for ubuntu-minimal https://review.openstack.org/505857 | 08:29 |
| *** hashar has quit IRC | 08:38 | |
| *** hashar has joined #openstack-dib | 08:39 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Add initramfs-tools for ubuntu-minimal https://review.openstack.org/505857 | 08:42 |
| openstackgerrit | Merged openstack/diskimage-builder master: Add initramfs-tools for ubuntu-minimal https://review.openstack.org/505857 | 08:44 |
| hwoarang | yolanda: should you add a proper reference to bug report on https://review.openstack.org/#/c/505857/ ? | 08:54 |
| yolanda | ah, the dib one... https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1700972 was linked | 08:56 |
| openstack | Launchpad bug 1700972 in linux-azure (Ubuntu Xenial) "Please only recommend or suggest initramfs-tools | linux-initramfs-tool for kernels able to boot without initramfs" [Low,Fix committed] - Assigned to Marcelo Cerri (mhcerri) | 08:56 |
| yolanda | anyway i already merged that, to try unlock the network problem as well. Hoping it works | 08:59 |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Support networkx 2.0 https://review.openstack.org/505963 | 09:43 |
| openstackgerrit | Merged openstack/diskimage-builder master: Support networkx 2.0 https://review.openstack.org/505963 | 09:43 |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: debootstrap: Call update-initramfs explicitly https://review.openstack.org/504588 | 09:44 |
| *** hashar has quit IRC | 10:16 | |
| *** jesusaur has quit IRC | 10:36 | |
| *** jesusaur has joined #openstack-dib | 10:40 | |
| *** andreas-f has quit IRC | 10:44 | |
| hwoarang | ianw: this https://review.openstack.org/#/c/505905/ pins networkx<2.0 but your https://review.openstack.org/#/c/505963/ dropped support for networkx-1.0 | 10:53 |
| hwoarang | and diskimage-builder doesn't have updated requirements.txt so it will not try to force networkx>2.0 | 10:54 |
| hwoarang | *>= | 10:54 |
| hwoarang | things will start breaking soon I believe | 10:54 |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Revert "Support networkx 2.0" https://review.openstack.org/506118 | 11:10 |
| yolanda | trying to revert the network>2.0 support... but things look bad with nodepool gateas | 11:17 |
| yolanda | gates | 11:17 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/506166 | 13:27 |
| *** chhavi has joined #openstack-dib | 13:31 | |
| *** hashar has joined #openstack-dib | 14:26 | |
| johnsom | http://logs.openstack.org/70/506170/1/check/gate-octavia-v1-dsvm-scenario-multinode/b3bbeb2/logs/devstacklog.txt.gz#_2017-09-21_14_13_09_539 | 14:31 |
| johnsom | Yep, broken with the g-r update | 14:32 |
| *** openstackgerrit has quit IRC | 14:33 | |
| *** andreas_s has joined #openstack-dib | 15:17 | |
| andreas_s | Hi is anyone facing an issue with the ubuntu minimal build? At least I do since yesterday. For the first thing a patch was already submitted https://review.openstack.org/#/c/505857/ | 15:18 |
| andreas_s | but now I'm facing a different one: https://bugs.launchpad.net/diskimage-builder/+bug/1718702 | 15:19 |
| openstack | Launchpad bug 1718702 in diskimage-builder "ubuntu minimal build fails with: KeyError: 'blockdev'" [Undecided,New] | 15:19 |
| andreas_s | (using version 2.8) | 15:19 |
| andreas_s | anyone facing the same? | 15:19 |
| *** andreas_s has quit IRC | 15:25 | |
| clarkb | andreas_s seems to have left but yes we were having kernel panics on xenial builds using ubuntu-minimal. ianw was looking at it iirc | 15:34 |
| hwoarang | clarkb: fyi https://bugs.launchpad.net/diskimage-builder/+bug/1718415 and perhaps this fixes it https://review.openstack.org/#/c/505857/ | 16:08 |
| openstack | Launchpad bug 1718415 in diskimage-builder "xenial image fails to mount rootfs" [Undecided,Confirmed] | 16:09 |
| clarkb | thanks | 16:09 |
| *** hashar has quit IRC | 16:18 | |
| *** Sukhdev has joined #openstack-dib | 17:57 | |
| *** chhavi has quit IRC | 17:59 | |
| rm_work | hey, any word on the strategy for fixing the networkx breakage? | 18:00 |
| rm_work | octavia gates are broken pending this issue | 18:00 |
| rm_work | I am willing to help if there's something I can do to speed this up | 18:00 |
| rm_work | do we know exactly what the plan is to fix it? moving forward with changes to use the new version, or pinning to an older one, or what? | 18:01 |
| rm_work | yolanda / hwoarang / ianw ^^ | 18:02 |
| *** Sukhdev has quit IRC | 18:03 | |
| yolanda | rm_work, i need reviews on https://review.openstack.org/506162 | 18:03 |
| rm_work | ok yeah just started looking at that | 18:03 |
| rm_work | did they really just rename a thing? | 18:03 |
| yolanda | they did a set of breaking changes when landing v2 | 18:04 |
| yolanda | i pinged ianw to review and approve it | 18:05 |
| yolanda | right now the dib code works for networkx 2.0 but fails for lower versions | 18:05 |
| rm_work | so what was dg.nodes is now dg.node, and there's a new thing that's different but named using the old name dg.nodes ? | 18:05 |
| rm_work | oh | 18:05 |
| rm_work | ok so | 18:05 |
| rm_work | kinda the opposite | 18:05 |
| rm_work | they moved what was "dg.node" to "dg.nodes"? | 18:06 |
| rm_work | ugh ok probably i am just distracting you | 18:06 |
| rm_work | is ianw the only one who can move this forward? | 18:06 |
| yolanda | this is the list of changes https://networkx.github.io/documentation/stable/release/migration_guide_from_1.x_to_2.0.html | 18:07 |
| yolanda | any other dib core could make it land as well | 18:07 |
| yolanda | or i could approve myself, but i don't think ian will take so much to join | 18:07 |
| rm_work | ok so i think I understand | 18:12 |
| rm_work | and I think your solution should work | 18:12 |
| johnsom | It looks like it failed a gate. | 18:14 |
| rm_work | nothing voting | 18:15 |
| rm_work | tripleo | 18:15 |
| rm_work | not sure if relevant | 18:15 |
| johnsom | Yeah, it was odd and looked like it was voting | 18:16 |
| rm_work | http://logs.openstack.org/62/506162/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/11abb24/console.html#_2017-09-21_13_29_23_867265 | 18:16 |
| rm_work | looks unrelated | 18:16 |
| rm_work | http://logs.openstack.org/62/506162/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/11abb24/console.html#_2017-09-21_13_27_34_525170 | 18:17 |
| rm_work | that task is probably well before actually doing the DIB stuff? | 18:17 |
| johnsom | http://logs.openstack.org/62/506162/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/11abb24/logs/undercloud/home/jenkins/repo_setup.log.txt.gz#_2017-09-21_13_27_37 | 18:19 |
| johnsom | Yeah, undercloud stuff, so I suspect not related | 18:20 |
| *** calebb has joined #openstack-dib | 19:38 | |
| calebb | hey, i seem to be running into https://bugs.launchpad.net/diskimage-builder/+bug/1718702 , any idea if there is some sort of temporary work around, or what might be causing the issue? | 19:42 |
| openstack | Launchpad bug 1718702 in diskimage-builder "ubuntu minimal build fails with: KeyError: 'blockdev'" [Undecided,New] | 19:42 |
| calebb | i tried using dib 2.7.2 instead of 2.8 and i had the same error | 19:42 |
| prometheanfire | I think I'm getting the same erorr on gentoo builds too | 19:46 |
| prometheanfire | I'll know in a couple min | 19:47 |
| prometheanfire | yep KeyError: 'blockdev' | 19:47 |
| prometheanfire | try reverting a47ff0dd4abbceae786d9a5a4d1ce8042a9eb4f7 (it's what I'm trying next | 19:48 |
| calebb | im gonna install from source and see if i can figure out a fix | 19:53 |
| bkero | Wow this got a lot more complicated than last time I looked | 19:55 |
| prometheanfire | actually that wone worked | 20:04 |
| prometheanfire | so master worksforme (at least for one image | 20:05 |
| bkero | prometheanfire: how about latest tag? | 20:06 |
| calebb | yeah im testing master and it looks like it might be working | 20:06 |
| prometheanfire | I run git master :| | 20:06 |
| prometheanfire | plus whatever patchset I'm working on | 20:06 |
| bkero | Time to bisect | 20:07 |
| prometheanfire | I think this is mostly a side effect of not running with upper-constraints | 20:08 |
| calebb | yeah master just worked for me too | 20:08 |
| prometheanfire | I wonder if dib has a 'is_truthy' function, so passed in envars could be '1,yes,Yes,true,True' etc | 20:30 |
| ianw | ok ... what's up :/ | 20:34 |
| prometheanfire | I think we are ok now | 20:36 |
| prometheanfire | https://bugs.launchpad.net/diskimage-builder/+bug/1718702 | 20:36 |
| openstack | Launchpad bug 1718702 in diskimage-builder "ubuntu minimal build fails with: KeyError: 'blockdev'" [Undecided,New] | 20:36 |
| prometheanfire | haven't reproduced since the multiversion networkx patch made it in | 20:36 |
| ianw | has that merged? | 20:40 |
| ianw | ok, so networkx is pinned to <2.0 in requirements now | 20:42 |
| ianw | and we've also merged a 2.0 fix https://review.openstack.org/505963 | 20:44 |
| ianw | and we have a bit of a hack out there to support both in https://review.openstack.org/#/c/506162 | 20:45 |
| ianw | an open critical tripleo bug in https://bugs.launchpad.net/tripleo/+bug/1718753/ | 20:45 |
| openstack | Launchpad bug 1718753 in tripleo "DIB fails to build images, UnboundLocalError: local variable 'call_order' referenced before assignment" [Critical,Triaged] - Assigned to Ian Wienand (iwienand) | 20:45 |
| *** openstackgerrit has joined #openstack-dib | 20:49 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/506166 | 20:49 |
| ianw | prometheanfire: 505975 ... i'd like to make sure we don't confuse ourselves even more :/ | 20:53 |
| ianw | ok, so it seems what should happen now is that we revert my 2.0 fix and take the <2.0 requirements pin from global requirements | 20:55 |
| ianw | then figure out how to make sure we're testing against upper constraints | 20:55 |
| ianw | then put the 2.0 change back against the requirements change to bump it, along with the other projects | 20:57 |
| ianw | ok, have to run for a bit but will do that, then look at a release | 21:00 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/506166 | 21:06 |
| openstackgerrit | Ben Nemec proposed openstack/diskimage-builder master: Revert "Support networkx 2.0" https://review.openstack.org/506376 | 21:11 |
| prometheanfire | ianw: sounds like a plan | 21:19 |
| prometheanfire | ianw: ok, so how about I also fix it in project-config? | 21:28 |
| *** Sukhdev has joined #openstack-dib | 21:28 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/506166 | 21:34 |
| clarkb | ianw: was the networkx issue the thing breaking the xenial build fix? | 21:34 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder master: Use upper-constraints in tox tests https://review.openstack.org/505975 | 21:35 |
| prometheanfire | ianw: https://review.openstack.org/505975 depends on the project-config change for dib to use upper-constraints, I think that's everywhere now? | 21:35 |
| clarkb | actually that change merged but failed CI? I am confused | 21:39 |
| prometheanfire | clarkb: aren't we all | 21:45 |
| ianw | clarkb: networkx was unrelated to xenial | 21:47 |
| *** evrardjp has quit IRC | 21:47 | |
| *** rm_work has quit IRC | 21:47 | |
| clarkb | ianw: ok, any idea how https://review.openstack.org/#/c/505857/7/ merged? it looks like CI was failing (due to networkx?) | 21:48 |
| ianw | clarkb: it says "change has been successfully pushed" ... maybe yolanda did it out of band? | 21:49 |
| clarkb | ya wondering if that is how gerrit shows that | 21:49 |
| tonyb | o/ | 21:49 |
| ianw | that said, i think what i want to do is pin everything to <2.0 networkx and then we can tackle doing the upgrade properly | 21:50 |
| ianw | tonyb: thanks, so my question is around how do i handle testing of new upper-constraints.txt with depends-on? | 21:50 |
| ianw | tonyb: like https://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/diskimage-builder.yaml#n35 | 21:50 |
| ianw | are there examples to follow? | 21:51 |
| tonyb | ianw: You need to rebase and take https://review.openstack.org/#/c/412747/ | 21:51 |
| tonyb | ianw: otherwise you don't support constrarints | 21:52 |
| tonyb | but once you have that then Depends-On shoudl do excatly what you want | 21:52 |
| *** evrardjp has joined #openstack-dib | 21:52 | |
| ianw | heh, 8 months ago :/ | 21:52 |
| tonyb | My fault as it hit a merge conflict and I didn't fix it | 21:53 |
| tonyb | Do you want me to do it now? | 21:53 |
| ianw | tonyb: ok, i'm just wondering about the non-tox case too though | 21:53 |
| ianw | i want to make sure testing is in sync | 21:54 |
| tonyb | I don't have a godd answer there, I normall just pip install -c giturl . | 21:55 |
| ianw | we shouldn't use a plain "pip install"? https://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/diskimage-builder.yaml#n35 should be a tox_install? | 21:55 |
| ianw | otherwise it seems like our functional tests will be different libraries to our unit tests, which seems ... confusing | 21:55 |
| tonyb | Yeah, I think the right way would be to also clone requirements (or use the macro?) which will set UPPER_CONSTRAINTS_FILE for you | 21:56 |
| tonyb | then the can pip install to pip install -c ..... | 21:57 |
| tonyb | Mind you your suggestion of just callign tox_oinstall shoudl work albeit with the potential for network issues to stomp on you | 21:57 |
| openstackgerrit | Tony Breeds proposed openstack/diskimage-builder master: Add Constraints support https://review.openstack.org/412747 | 21:59 |
| ianw | that would make sense. tl;dr -- add requirements to the zuul-cloner, and use "-c" in the pip install pointing at that ? | 21:59 |
| tonyb | Yup | 22:00 |
| ianw | ok | 22:01 |
| tonyb | ianw: I don't know if you can just use: https://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n40 | 22:02 |
| *** rm_work has joined #openstack-dib | 22:02 | |
| *** rm_work has quit IRC | 22:02 | |
| *** rm_work has joined #openstack-dib | 22:02 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Revert "Support networkx 2.0" https://review.openstack.org/506398 | 22:03 |
| tonyb | and then just pip install -c ./upper-constraints.txt .... | 22:03 |
| tonyb | basically the same thing just less grow your own | 22:03 |
| ianw | prometheanfire / tonyb: ^ so we're in agreement on the broad steps in 506398 | 22:03 |
| ianw | i will do a release with the <2.0 pin. then we can add proper constraints support | 22:04 |
| ianw | tonyb: oh nice, yes that would be the way to go | 22:04 |
| *** owalsh_ has joined #openstack-dib | 22:04 | |
| tonyb | ianw: +1 I endorse that plan | 22:05 |
| prometheanfire | I've been distracted, what's up? | 22:06 |
| ianw | tonyb: ok, thanks :) | 22:06 |
| ianw | prometheanfire: basically the steps in 506398 | 22:06 |
| *** owalsh- has joined #openstack-dib | 22:06 | |
| ianw | with step 1.5 being a 2.9 dib release so people can use that | 22:07 |
| *** owalsh has quit IRC | 22:07 | |
| *** owalsh- is now known as owalsh | 22:07 | |
| prometheanfire | looks like https://review.openstack.org/506398 is step 1 | 22:07 |
| prometheanfire | and 1a | 22:07 |
| clarkb | ianw: once you'd got things settled, can you make sure to let us know when you think it is safe to build xenial images again? | 22:07 |
| ianw | clarkb: yep, that will be when 2.9 release is made | 22:08 |
| prometheanfire | https://review.openstack.org/505975 with it's depends-on should be step 2 | 22:08 |
| clarkb | cool, thanks | 22:08 |
| ianw | prometheanfire: i think we want 412747? | 22:09 |
| *** owalsh_ has quit IRC | 22:09 | |
| ianw | btw, this is mea culpa ... i should have thought ahead about what happened when an incompatible 2.0 was released when i was warned about it with that bug | 22:09 |
| tonyb | prometheanfire: personally I think https://review.openstack.org/#/c/412747/ not 505975 | 22:09 |
| ianw | live and learn | 22:10 |
| tonyb | prometheanfire: the depends-on is fine but slightly sub-optimal | 22:11 |
| tonyb | anyway /me needs linner | 22:11 |
| prometheanfire | tonyb: right, forgot about the self refrence thing | 22:11 |
| prometheanfire | tonyb: you need to update that patch, it include a merge conflict (was auto --contined it looks like) | 22:12 |
| tonyb | Gah how did I do that? | 22:12 |
| prometheanfire | tonyb: guess I'll need to figure out how to fix it up in project-config then | 22:13 |
| prometheanfire | we still need something like https://review.openstack.org/#/c/506388/ | 22:13 |
| prometheanfire | the easiest way would be to just sed out the disimagebuilder line from upper-constraints | 22:14 |
| openstackgerrit | Tony Breeds proposed openstack/diskimage-builder master: Add Constraints support https://review.openstack.org/412747 | 22:15 |
| ianw | prometheanfire: ohh, so "pip install -cFOO -e ." in the diskimage-builder directory is going to look at the constraints, see that the local dir is higher and take the pip version? | 22:17 |
| ianw | or does it figure this out? | 22:17 |
| clarkb | constraints always wins. | 22:18 |
| clarkb | so if you have foo>3.0 in requirements but foo===2.5 in constraints then 2.5 will be installed | 22:18 |
| prometheanfire | yep | 22:18 |
| *** yolanda has quit IRC | 22:18 | |
| prometheanfire | that's what it needs the sed | 22:18 |
| ianw | clarkb: even if you're installing from your git checkout? | 22:19 |
| prometheanfire | I just updated https://review.openstack.org/506388 | 22:19 |
| prometheanfire | yep | 22:19 |
| prometheanfire | that's the === power | 22:19 |
| clarkb | ya, which is why we have the special machinery to work around that | 22:19 |
| prometheanfire | :D | 22:19 |
| ianw | how to other projects deal with that? | 22:19 |
| *** yolanda has joined #openstack-dib | 22:20 | |
| prometheanfire | that's what https://review.openstack.org/#/c/412747/4/tools/tox_install.sh fixes | 22:20 |
| prometheanfire | also not sure if my sed method is the correct one for the project-config change though | 22:21 |
| ianw | ahhhh, ok | 22:22 |
| ianw | prometheanfire: why not use edit-constraints? | 22:22 |
| prometheanfire | complexity mostly | 22:22 |
| ianw | i think the complexity horse has bolted :) | 22:23 |
| ianw | basically i guess make the project-config script look like tox_install.sh | 22:23 |
| prometheanfire | well, I'm trying to not let it get too far :P | 22:23 |
| prometheanfire | I could do that, but don't think it's necessarty | 22:24 |
| prometheanfire | I could do that, but don't think it's necessary | 22:24 |
| ianw | yeah, all this will need to be fiddled for zuulv3 ... but one thing at a time | 22:29 |
| ianw | but it is why i'd like to use as much generic infrastructure as possible, so that we fit closer to any migration plans everyone else has | 22:30 |
| prometheanfire | ya | 22:30 |
| ianw | ok, i'm out for a bit ... we're just waiting for the revert/pin to land, then i'll quick approve that and go for 2.9.0 release | 22:48 |
| ianw | then we can consider all the other testing stuff :) | 22:48 |
| *** Sukhdev has quit IRC | 22:57 | |
| * prometheanfire is just sitting working on gentoo things | 23:41 | |
| prometheanfire | btw, I'm redoing the gentoo patch, the python update is going to be first things, because some things could be pip installed with a python version that is later removed (aka simple-init/glean) | 23:42 |
| *** chhavi has joined #openstack-dib | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!