Thursday, 2018-02-15

*** hshiina|afk is now known as hshiina00:08
*** jcoufal_ has joined #openstack-ironic00:12
*** jcoufal has quit IRC00:12
rybridgesHello. I had a quick question: Is the "updated" field on an Ironic node editable via the Python client? I recall someone telling me at one point that it is immutable00:13
jrollrybridges: no, that's updated whenever the database record for the node is updated00:14
jrollyou could update any other field to make it update :P00:14
rybridgesThought so. Thanks jroll00:16
*** jcoufal has joined #openstack-ironic00:16
jrollnp00:16
*** jcoufal_ has quit IRC00:18
rybridgesjroll does that also apply to updating arbitrary node properties?00:19
jrollrybridges: 99% sure, yeah00:20
*** rloo has quit IRC00:20
penickrybridges the updated_at field is an auto field in the DB schema00:20
*** jcoufal_ has joined #openstack-ironic00:22
jrollah yes, this00:22
*** jcoufal has quit IRC00:23
*** jcoufal has joined #openstack-ironic00:26
openstackgerritJulia Kreger proposed openstack/ironic master: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54475000:28
* TheJulia crosses her fingers00:28
*** jcoufal_ has quit IRC00:28
jroll\o/00:28
* TheJulia goes and puts a dress on... because you know... dinnner00:28
jrollTheJulia: damnit, I gave you a bad command00:28
* jroll updates00:28
TheJuliadoh!00:29
TheJuliait looked good at a super quick glance00:29
TheJuliaoh, yeah I see it now00:29
jrollTheJulia: also, does this run as root?00:30
jrollI feel like it doesn't00:30
*** jcoufal_ has joined #openstack-ironic00:30
openstackgerritJim Rollenhagen proposed openstack/ironic master: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54475000:31
* jroll assumes it does, sees what happens00:31
*** jcoufal has quit IRC00:32
*** jcoufal has joined #openstack-ironic00:34
*** jcoufal_ has quit IRC00:36
*** jcoufal_ has joined #openstack-ironic00:39
*** jcoufal has quit IRC00:40
openstackgerritMerged openstack/ironic-inspector master: Retry port lists on failure in PXE filter periodic sync  https://review.openstack.org/54392800:40
*** jcoufal has joined #openstack-ironic00:42
*** jcoufal_ has quit IRC00:44
*** jcoufal_ has joined #openstack-ironic00:45
*** actatux has quit IRC00:46
*** Goneri has quit IRC00:47
*** jcoufal has quit IRC00:48
*** jcoufal has joined #openstack-ironic00:50
*** jcoufal_ has quit IRC00:52
*** jcoufal_ has joined #openstack-ironic00:54
openstackgerritJulia Kreger proposed openstack/ironic master: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54475000:54
TheJuliajroll: nope, doesn't... but that might work00:55
*** jcoufal has quit IRC00:56
*** rpioso is now known as rpioso|afk00:56
*** r-daneel has quit IRC00:56
*** actatux has joined #openstack-ironic00:59
*** mjturek has quit IRC01:04
openstackgerritHironori Shiina proposed openstack/ironic master: Update a CI job for rescue mode  https://review.openstack.org/52870401:07
TheJuliahmm... that may have done it01:10
* TheJulia crosses her fingers01:10
*** jcoufal_ has quit IRC01:20
*** jcoufal_ has joined #openstack-ironic01:22
*** jcoufal has joined #openstack-ironic01:25
*** jcoufal_ has quit IRC01:27
openstackgerritJulia Kreger proposed openstack/ironic master: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54475001:28
*** jcoufal_ has joined #openstack-ironic01:31
*** jcoufal has quit IRC01:32
*** jcoufal has joined #openstack-ironic01:34
openstackgerritMerged openstack/ironic master: Updated from global requirements  https://review.openstack.org/54426301:34
openstackgerritMerged openstack/ironic master: Update to standalone ironic doc  https://review.openstack.org/54421501:34
*** jcoufal_ has quit IRC01:36
*** jcoufal_ has joined #openstack-ironic01:38
*** jcoufal has quit IRC01:40
openstackgerritMerged openstack/ironic master: Switch contributor documentation to hardware types  https://review.openstack.org/53795901:42
*** jcoufal has joined #openstack-ironic01:42
*** jcoufal_ has quit IRC01:45
*** jcoufal_ has joined #openstack-ironic01:46
*** jcoufal has quit IRC01:48
*** jcoufal has joined #openstack-ironic01:50
*** jcoufal_ has quit IRC01:52
*** jcoufal_ has joined #openstack-ironic01:53
*** jcoufal has quit IRC01:56
*** jcoufal has joined #openstack-ironic01:58
*** jcoufal_ has quit IRC02:00
*** jcoufal_ has joined #openstack-ironic02:02
*** jcoufal has quit IRC02:04
*** jcoufal has joined #openstack-ironic02:06
*** jcoufal_ has quit IRC02:08
*** jcoufal_ has joined #openstack-ironic02:09
*** jcoufal has quit IRC02:12
*** khyr0n has joined #openstack-ironic02:46
*** jcoufal has joined #openstack-ironic02:58
*** jcoufal_ has quit IRC03:00
*** jcoufal_ has joined #openstack-ironic03:06
*** jcoufal has quit IRC03:08
*** harlowja has quit IRC03:08
*** jcoufal has joined #openstack-ironic03:11
khyr0nHi guys, have a little problem with tripleo ironic... basically the introspection works perfectly, but if I try to deploy the cloud, the baremetal node never gets and answer from the dhcp server03:11
*** jcoufal_ has quit IRC03:12
*** ijw has quit IRC03:13
*** jcoufal_ has joined #openstack-ironic03:14
*** jcoufal has quit IRC03:16
*** jcoufal has joined #openstack-ironic03:26
*** jcoufal_ has quit IRC03:28
*** ijw has joined #openstack-ironic03:31
*** jcoufal_ has joined #openstack-ironic03:31
*** jcoufal has quit IRC03:32
*** jcoufal has joined #openstack-ironic03:34
*** ijw has quit IRC03:35
*** jcoufal_ has quit IRC03:36
*** ijw has joined #openstack-ironic03:43
*** jcoufal_ has joined #openstack-ironic03:46
*** jcoufal has quit IRC03:48
*** ijw has quit IRC03:48
*** jcoufal has joined #openstack-ironic03:50
*** jcoufal_ has quit IRC03:52
*** jcoufal_ has joined #openstack-ironic03:55
*** jcoufal has quit IRC03:58
*** jcoufal has joined #openstack-ironic04:01
*** jcoufal_ has quit IRC04:04
*** jcoufal_ has joined #openstack-ironic04:07
*** jcoufal has quit IRC04:08
*** jcoufal has joined #openstack-ironic04:10
*** khyr0n has quit IRC04:11
*** Nisha_Agarwal has joined #openstack-ironic04:12
*** khyr0n has joined #openstack-ironic04:12
*** jcoufal_ has quit IRC04:12
*** jcoufal_ has joined #openstack-ironic04:13
*** exodusftw has quit IRC04:15
*** jcoufal has quit IRC04:16
*** devananda has quit IRC04:17
*** khyr0n has quit IRC04:17
openstackgerritMallikarjuna Reddy Kolagatla proposed openstack/ironic master: Updates boot mode on the baremetal as per `boot_mode`  https://review.openstack.org/54079304:19
*** jcoufal has joined #openstack-ironic04:19
*** jcoufal_ has quit IRC04:20
*** exodusftw has joined #openstack-ironic04:21
*** jcoufal_ has joined #openstack-ironic04:21
*** ijw has joined #openstack-ironic04:21
*** jaganathan has joined #openstack-ironic04:22
*** jcoufal has quit IRC04:24
*** ijw has quit IRC04:26
*** jcoufal has joined #openstack-ironic04:26
openstackgerritMallikarjuna Reddy Kolagatla proposed openstack/ironic master: Updates boot mode on the baremetal as per `boot_mode`  https://review.openstack.org/54079304:27
*** jcoufal_ has quit IRC04:28
*** jcoufal_ has joined #openstack-ironic04:30
*** jcoufal has quit IRC04:32
*** masber has joined #openstack-ironic04:33
*** links has joined #openstack-ironic04:34
*** jcoufal has joined #openstack-ironic04:34
*** jcoufal_ has quit IRC04:36
*** jcoufal_ has joined #openstack-ironic04:38
*** jcoufal has quit IRC04:40
*** jcoufal has joined #openstack-ironic04:41
*** links has quit IRC04:42
*** jcoufal_ has quit IRC04:44
*** links has joined #openstack-ironic04:45
*** jcoufal_ has joined #openstack-ironic04:45
*** jcoufal has quit IRC04:46
*** dpawar has joined #openstack-ironic04:47
*** jcoufal has joined #openstack-ironic04:48
*** jcoufal_ has quit IRC04:50
*** jcoufal_ has joined #openstack-ironic04:52
*** jcoufal has quit IRC04:54
*** jcoufal has joined #openstack-ironic04:56
*** r-daneel has joined #openstack-ironic04:57
*** jcoufal_ has quit IRC04:58
*** jcoufal_ has joined #openstack-ironic05:01
*** jcoufal has quit IRC05:03
*** jcoufal has joined #openstack-ironic05:09
*** threestrands_ has joined #openstack-ironic05:09
*** threestrands_ has quit IRC05:09
*** threestrands_ has joined #openstack-ironic05:09
*** threestrands has quit IRC05:09
*** jcoufal_ has quit IRC05:10
*** threestrands_ has quit IRC05:10
*** threestrands_ has joined #openstack-ironic05:11
*** jcoufal_ has joined #openstack-ironic05:12
*** jcoufal has quit IRC05:13
*** harlowja has joined #openstack-ironic05:15
*** threestrands has joined #openstack-ironic05:15
*** threestrands has quit IRC05:15
*** threestrands has joined #openstack-ironic05:15
*** threestrands_ has quit IRC05:18
*** jcoufal has joined #openstack-ironic05:23
*** jcoufal_ has quit IRC05:24
*** jcoufal_ has joined #openstack-ironic05:26
*** threestrands has quit IRC05:28
*** jcoufal has quit IRC05:29
*** ccollett has joined #openstack-ironic05:35
*** jcoufal has joined #openstack-ironic05:36
*** charcol-v2 has quit IRC05:38
*** jcoufal_ has quit IRC05:39
*** jcoufal_ has joined #openstack-ironic05:39
*** jcoufal has quit IRC05:42
*** jcoufal has joined #openstack-ironic05:44
*** jcoufal_ has quit IRC05:46
*** jcoufal_ has joined #openstack-ironic05:48
*** jcoufal has quit IRC05:50
*** jcoufal has joined #openstack-ironic05:52
*** jcoufal_ has quit IRC05:54
*** jcoufal_ has joined #openstack-ironic05:57
*** jcoufal has quit IRC05:59
*** jcoufal has joined #openstack-ironic06:00
*** jcoufal_ has quit IRC06:02
*** jcoufal_ has joined #openstack-ironic06:12
*** jcoufal has quit IRC06:13
*** harlowja has quit IRC06:14
*** jcoufal has joined #openstack-ironic06:17
*** jcoufal_ has quit IRC06:18
*** marios has joined #openstack-ironic06:18
*** jcoufal_ has joined #openstack-ironic06:19
*** e0ne has joined #openstack-ironic06:20
*** jcoufal has quit IRC06:22
*** jcoufal has joined #openstack-ironic06:25
Tenguhello there! small question: is there a way to inject a node MAC address once it's registered in ironic?06:25
*** jcoufal_ has quit IRC06:26
*** jcoufal_ has joined #openstack-ironic06:27
*** jcoufal has quit IRC06:29
*** e0ne has quit IRC06:30
*** jcoufal has joined #openstack-ironic06:30
*** jcoufal_ has quit IRC06:33
*** jcoufal_ has joined #openstack-ironic06:34
*** mjura has joined #openstack-ironic06:35
*** jcoufal has quit IRC06:36
*** rcernin has quit IRC06:44
*** e0ne has joined #openstack-ironic06:44
*** jcoufal has joined #openstack-ironic06:45
*** jcoufal_ has quit IRC06:47
*** jcoufal_ has joined #openstack-ironic06:49
*** jcoufal has quit IRC06:50
*** e0ne_ has joined #openstack-ironic06:51
*** e0ne_ has quit IRC06:55
*** e0ne has quit IRC06:55
*** etingof has quit IRC06:55
*** bfournie has quit IRC06:55
*** hoonetorg has quit IRC06:56
*** e0ne has joined #openstack-ironic06:56
*** hoonetorg has joined #openstack-ironic06:57
*** jcoufal has joined #openstack-ironic07:02
*** jcoufal_ has quit IRC07:02
*** olivierbourdon38 has quit IRC07:06
*** etingof has joined #openstack-ironic07:07
*** hshiina has quit IRC07:09
*** threestrands has joined #openstack-ironic07:09
*** threestrands has quit IRC07:09
*** threestrands has joined #openstack-ironic07:09
*** hshiina has joined #openstack-ironic07:09
*** jcoufal_ has joined #openstack-ironic07:17
*** jcoufal has quit IRC07:19
*** rbartal has joined #openstack-ironic07:25
*** ccollett has quit IRC07:28
*** dpawar has quit IRC07:28
*** rcernin has joined #openstack-ironic07:28
*** pcaruana has joined #openstack-ironic07:40
*** e0ne has quit IRC07:43
*** e0ne has joined #openstack-ironic07:43
*** e0ne has quit IRC07:49
*** dpawar has joined #openstack-ironic07:53
*** jcoufal has joined #openstack-ironic07:57
*** jcoufal_ has quit IRC07:59
*** jcoufal_ has joined #openstack-ironic08:02
*** MattMan has quit IRC08:02
*** MattMan has joined #openstack-ironic08:02
*** jcoufal has quit IRC08:04
*** AlexeyAbashkin has joined #openstack-ironic08:08
*** ijw has joined #openstack-ironic08:17
*** tesseract has joined #openstack-ironic08:17
*** olivierbourdon38 has joined #openstack-ironic08:19
*** ijw has quit IRC08:21
*** pmannidi has quit IRC08:22
*** links has quit IRC08:35
openstackgerritMerged openstack/bifrost master: Fix DHCP informations according to network interface settings  https://review.openstack.org/54216708:36
Tenguok, found it, we can update the baremetal port via the following command: openstack baremetal port set --address MAC PORT_UUID08:39
*** mgoddard_ has joined #openstack-ironic08:44
*** Nisha_Agarwal has quit IRC08:44
*** jaianshu has joined #openstack-ironic08:50
openstackgerritMerged openstack/bifrost master: Fix for proper usage when keystone enabled  https://review.openstack.org/54344108:50
*** links has joined #openstack-ironic08:52
*** Nisha_Agarwal has joined #openstack-ironic09:01
*** sambetts|afk has quit IRC09:03
*** sambetts_ has joined #openstack-ironic09:07
*** derekh has joined #openstack-ironic09:12
*** jcoufal has joined #openstack-ironic09:16
*** jcoufal_ has quit IRC09:18
*** jcoufal_ has joined #openstack-ironic09:20
*** dtantsur|afk is now known as dtantsur09:21
*** threestrands_ has joined #openstack-ironic09:21
*** threestrands has quit IRC09:21
*** jcoufal has quit IRC09:23
*** vsaienk0 has joined #openstack-ironic09:23
olivierbourdon38yolanda thx a lot for the merges09:25
*** hshiina is now known as hshiina|afk09:25
yolandayou're welcome, really nice job there09:25
openstackgerritDmitry Tantsur proposed openstack/ironic master: tox: stop validating locale files  https://review.openstack.org/54352309:30
dtantsurmorning ironic09:31
openstackgerritDmitry Tantsur proposed openstack/ironic master: Imported Translations from Zanata  https://review.openstack.org/54260909:32
*** jcoufal has joined #openstack-ironic09:33
*** vsaienk0 has quit IRC09:33
*** ijw has joined #openstack-ironic09:34
*** jcoufal_ has quit IRC09:35
*** jcoufal_ has joined #openstack-ironic09:36
Nisha_Agarwaldtantsur, morning....There is a document patch where we deprecate the classical ilo drivers. https://review.openstack.org/54259309:36
patchbotpatch 542593 - ironic - Update iLO documentation for deprecating classical...09:36
*** e0ne has joined #openstack-ironic09:37
dtantsurNisha_Agarwal: yep, thanks, it's on my radar09:38
dtantsurjust fixing one nasty bug first09:38
Nisha_Agarwaldtantsur, oh thanks.09:38
*** jcoufal has quit IRC09:38
Nisha_Agarwaldtantsur, when is the ironic release targetted?09:38
Nisha_Agarwalfor Q09:38
*** ijw has quit IRC09:39
Nisha_Agarwalthis week? or next week?09:39
dtantsurNisha_Agarwal: we are planning on another queens release next week. but better ask TheJulia, she's the PTL now :)09:39
Nisha_Agarwal:)09:39
Nisha_Agarwaloh good.09:39
Nisha_AgarwalTheJulia, congrats09:39
Nisha_Agarwali joined late this weekly meeting. i think i missed that announcement then09:40
Nisha_Agarwal:)09:40
Nisha_Agarwaldtantsur, hope you would be still with ironic09:42
Nisha_Agarwal:)09:42
dtantsurNisha_Agarwal: sure, I'm not going anywhere :) at least not in the near future09:42
*** jcoufal has joined #openstack-ironic09:59
*** hshiina|afk has quit IRC10:01
*** jcoufal_ has quit IRC10:01
*** jcoufal_ has joined #openstack-ironic10:02
*** jcoufal has quit IRC10:05
*** jcoufal has joined #openstack-ironic10:06
*** jcoufal_ has quit IRC10:08
*** vsaienk0 has joined #openstack-ironic10:09
*** jcoufal_ has joined #openstack-ironic10:10
olivierbourdon38Sorry for the basic questions but I currently have a hard time understanding the ironic/bifrost "workflow" and reading the docs and searching the web currently did not help so here are my setup and questions10:11
olivierbourdon38I have deployed bifrost in a kvm VM which also communicates with vbmc on the host running the VM to enable ipmi like support10:12
*** jcoufal has quit IRC10:13
olivierbourdon38booting another VM boots the small linux image containing IPA and this works well and I see the node in the ironic node-list output10:13
olivierbourdon38doing a validate shows that the ipmi credentials are ok and that I can properly get the state of the VM10:14
olivierbourdon38using ipmitool also shows that I can power cycle the VM properly10:14
olivierbourdon38so far so good10:14
olivierbourdon38in the default bifrost deployment the /httpboot folder contains the IPA image and a dib build debian basic image named deployment_image.qcow210:16
*** Nisha_Agarwal has quit IRC10:17
olivierbourdon38what are the next node configuration steps to be able to move the node state into manage -> provide -> deploy and get my VM provisioned ?10:17
olivierbourdon38I think it must have to do with the --driver-info deploy_kernel=$kernel --driver-info deploy_ramdisk=$ramdisk and/or --instance-info image_source=http://192.168.1.254:8080/deployment_image.qcow2 --instance-info image_checksum=... but my experiments failed so far10:18
olivierbourdon38thx for your understanding10:18
olivierbourdon38and help10:18
*** vsaienk0 has quit IRC10:19
dtantsurolivierbourdon38: hi! have you seen https://docs.openstack.org/ironic/latest/install/standalone.html ?10:19
*** sambetts_ is now known as sambetts10:21
sambettsolivierbourdon38: if you are using bifrost, bifrost provides a playbook for deploying the node which will populate all the right information in ironic for you10:22
openstackgerritIlya Etingof proposed openstack/ironic master: Raw image size estimation improved  https://review.openstack.org/54483910:23
olivierbourdon38dtantsur yep I did and indeed I have a node validate output which seems correct10:24
*** jcoufal has joined #openstack-ironic10:24
olivierbourdon38however the deploy seems to require either/both other parameters like kernel/amdisk/image+checksum but as I only have a single qcow2 image seems like the choice is reduced and deploy fails10:26
olivierbourdon38I must be missing something very obvious10:26
*** jcoufal_ has quit IRC10:27
openstackgerritMerged openstack/ironic-ui stable/queens: Imported Translations from Zanata  https://review.openstack.org/54420010:27
sambettsdriver_info kernal/ramdisk is for providing the IPA image10:27
sambettsused for deploying10:28
sambettsthe instance-info field is used for providing the information about hte image to be deployed10:28
olivierbourdon38ok, that makes sense so I only need the instance-info image10:28
olivierbourdon38sambetts thx10:28
sambettsyou'll need to have the deploy info fields set to the IPA image otherwise ironic won't be able to boot the ramdisk10:29
sambettsbut that is normally an enrol time configuruation10:29
sambettsnot a deploy time configuration10:29
olivierbourdon38enrolling is ok and I get my VM in node list with enroll status10:30
*** d0ugal has quit IRC10:30
*** d0ugal has joined #openstack-ironic10:33
*** jcoufal_ has joined #openstack-ironic10:38
*** jcoufal has quit IRC10:40
*** jcoufal has joined #openstack-ironic10:43
*** jcoufal_ has quit IRC10:45
*** jcoufal_ has joined #openstack-ironic10:47
*** jcoufal has quit IRC10:49
*** jcoufal has joined #openstack-ironic10:50
*** e0ne has quit IRC10:51
*** jcoufal_ has quit IRC10:52
*** jaganathan has quit IRC10:54
*** jcoufal_ has joined #openstack-ironic10:55
*** jcoufal has quit IRC10:57
*** jcoufal has joined #openstack-ironic10:59
*** priteau has joined #openstack-ironic10:59
*** e0ne has joined #openstack-ironic10:59
*** jistr is now known as jistr|mtg11:00
*** jcoufal_ has quit IRC11:00
olivierbourdon38analyzing deeper the traffic over the network interface seems like IPA fails to retrieve GET /v1/lookup?addresses=52%3A54%3A00%3Aec%3A01%3Ae4 HTTP/1.1\r\n -> 40411:02
*** jcoufal_ has joined #openstack-ironic11:03
sambettshave you populated ironic with the port information for your baremetal node?11:03
*** lucas-afk is now known as lucasagomes11:04
*** jcoufal has quit IRC11:05
olivierbourdon38the IPA has done this at 1st boot leading to the enroll state 1 port with proper MAC addr attached to node11:09
*** jaganathan has joined #openstack-ironic11:11
*** jcoufal has joined #openstack-ironic11:13
*** vsaienk0 has joined #openstack-ironic11:14
*** jcoufal_ has quit IRC11:15
*** jcoufal_ has joined #openstack-ironic11:16
olivierbourdon38https://gist.github.com/obourdon/9cdb807c882ab5b03ce1bf0e5790f6c511:17
sambettsolivierbourdon38: ok so that looks correct, whats the state of your node?11:19
sambettsalso which version of ironic are you using??11:19
*** jcoufal has quit IRC11:19
*** d0ugal has quit IRC11:20
olivierbourdon38ironic --version11:21
olivierbourdon38The "ironic" CLI is deprecated and will be removed in the S* release. Please use the "openstack baremetal" CLI instead.11:21
olivierbourdon382.2.011:21
olivierbourdon38for the CLI11:21
olivierbourdon38and the default which comes with yesteday's bifrost master11:21
olivierbourdon38for the other parts11:22
*** e0ne has quit IRC11:24
*** vsaienk0 has quit IRC11:25
sambettsolivierbourdon38: hmm can you check the ironic API logs to see if there was any reason given for not finding the node by the mac address?11:26
*** e0ne has joined #openstack-ironic11:31
olivierbourdon38see comment in gist entry above, nothing really helpfull, may be I should activate debug level tracing ?11:39
*** d0ugal has joined #openstack-ironic11:39
*** jaganathan has quit IRC11:42
olivierbourdon38Feb 15 12:44:06 master.vagrantup.com ironic-api[11398]: 2018-02-15 12:44:06.867 11408 DEBUG wsme.api [req-4dd2a6d0-7b92-48f2-96c5-ab8266f225b9 - - - - -] Client-side error: Resource could not be found. format_exception /usr/local/lib/python2.7/dist-packages/wsme/api.py:22211:45
olivierbourdon38no sure it is releated though11:45
*** jcoufal has joined #openstack-ironic11:46
*** jcoufal_ has quit IRC11:48
*** jcoufal_ has joined #openstack-ironic11:51
openstackgerritMerged openstack/ironic-inspector stable/queens: Retry port lists on failure in PXE filter periodic sync  https://review.openstack.org/54472111:51
openstackgerritMerged openstack/ironic stable/queens: Switch contributor documentation to hardware types  https://review.openstack.org/54472611:51
*** jcoufal has quit IRC11:53
*** jaganathan has joined #openstack-ironic11:59
openstackgerritMerged openstack/ironic master: Remove too large configdrive for handling error  https://review.openstack.org/54209712:02
openstackgerritMerged openstack/ironic-python-agent master: Follow-up patch for documentation of rescue mode  https://review.openstack.org/54233912:02
openstackgerritMerged openstack/python-ironicclient stable/pike: Zuul: Remove project name  https://review.openstack.org/54402412:02
*** jcoufal has joined #openstack-ironic12:02
openstackgerritDmitry Tantsur proposed openstack/ironic stable/queens: Remove too large configdrive for handling error  https://review.openstack.org/54487712:03
*** nicodemos has joined #openstack-ironic12:03
*** jcoufal_ has quit IRC12:05
*** jcoufal_ has joined #openstack-ironic12:07
*** jcoufal has quit IRC12:09
*** ricardoas has joined #openstack-ironic12:14
*** jcoufal_ has quit IRC12:14
*** dpawar has quit IRC12:21
*** jistr|mtg is now known as jistr12:23
olivierbourdon38did not mentioned this earlier but please note that I also have keystone enabled12:24
jrollmorning folks12:25
dtantsurhey jroll12:25
jrollTheJulia: I think we totally broke ansible with that :D12:26
jrollmight need to be a one-liner, I guess12:27
dtantsurjroll: broken ansible with what? :)12:29
jrolldtantsur: so our theory on grenade is that a .pyc file is getting updated when we upgrade ironic, and then breaking nova12:29
jrollso here's an attempt to prove that https://review.openstack.org/#/c/54475012:29
patchbotpatch 544750 - ironic - DNM Test disabling .pyc files for Grenade12:29
* jroll updates12:29
*** e0ne has quit IRC12:29
dtantsurwow12:30
*** dpawar has joined #openstack-ironic12:30
jrollobviously not a solution, but will give us some info :)12:31
openstackgerritJim Rollenhagen proposed openstack/ironic master: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54475012:33
*** dpawar has quit IRC12:34
*** jcoufal has joined #openstack-ironic12:40
*** bfournie has joined #openstack-ironic12:40
*** bfournie has quit IRC12:40
*** bfournie has joined #openstack-ironic12:41
*** mgoddard_ has quit IRC12:42
*** bfournie has quit IRC12:45
*** cdearborn has joined #openstack-ironic12:57
*** dprince has joined #openstack-ironic13:00
*** jaianshu has quit IRC13:02
*** jcoufal_ has joined #openstack-ironic13:13
*** jcoufal has quit IRC13:15
*** bfournie has joined #openstack-ironic13:17
*** mgoddard_ has joined #openstack-ironic13:21
*** trown|outtypewww is now known as trown13:23
*** jcoufal has joined #openstack-ironic13:24
*** jcoufal_ has quit IRC13:25
olivierbourdon38any pointers/hints to share with me on how to debug the ironic api (at first) and other processes ? Which python source files to look at ? any kind of bootstrapping greatly appreciated13:33
*** jcoufal_ has joined #openstack-ironic13:38
jrollolivierbourdon38: ironic/api/controllers/ should have most of the API code, things that RPC to the conductor go through ironic/conductor/rpcapi.py and then to ironic/conductor/manager.py, where the bulk of the work is done13:38
olivierbourdon38jroll thanks will have a look right away13:38
jrolland yes, debug logging would likely help you out13:38
*** jcoufal has quit IRC13:40
*** jcoufal has joined #openstack-ironic13:41
*** Goneri has joined #openstack-ironic13:42
*** etingof has quit IRC13:43
*** jcoufal_ has quit IRC13:44
*** e0ne has joined #openstack-ironic13:54
*** etingof has joined #openstack-ironic13:57
*** cdearborn has quit IRC14:02
*** dtantsur is now known as dtantsur|brb14:02
*** mjturek has joined #openstack-ironic14:03
jrollthis grenade test patch has most of our vendors failing it, with no change to any code they run :(14:05
*** jcoufal_ has joined #openstack-ironic14:05
* TheJulia tries to wake up14:05
sambettsjroll: I'm seeing this on that patchin our CI http://192.133.156.18/50/544750/5/check/ironic-dsvm-tempest-ironic-cimc-current-centos-7-cimc/c4ec784/logs/devstacklog.txt.gz#_2018-02-15_14_01_04_09914:06
*** vsaienk0 has joined #openstack-ironic14:06
TheJuliajroll: yeah, we did it seems :\14:06
jrollsambetts: oh, I saw someone else asking about that this mornign :/14:07
jrollprobably deleted from pypi, wonder if that's affecting our upstream CI as well14:07
*** jcoufal has quit IRC14:07
jrolldoesn't look like it14:07
jrollinfra caches, I guess14:08
sambettsmust be... thats worring though...14:08
*** jcoufal has joined #openstack-ironic14:08
jrollwhich bit, that upstream doesn't start breaking in these cases?14:08
sambettsthat something in uppercontraits points to a deleted version14:09
sambettsand its not being caught by upstream14:09
jrollyeah, it's not awesome14:09
*** lucasagomes is now known as lucas-hungry14:09
jrollTheJulia: hm, even the one liner still breaks it, lame14:10
* jroll wonders how else to best inject that14:10
sambettsjroll: just checked, same error in all the vendor CIs on that patch14:10
TheJuliajroll: yeah, I was just looking at that :(14:10
*** jcoufal_ has quit IRC14:11
jrollTheJulia: actually... we got much further into the job than our hack14:12
jrollpip install seems to work fine14:13
*** jcoufal_ has joined #openstack-ironic14:13
TheJuliayeah, I noticed :\14:13
sambettssuper annoying that the infra ansible callback plugin removes the task/role names from the logs14:13
jrolleh, if you go here you can see what failed: http://logs.openstack.org/50/544750/5/check/ironic-grenade-dsvm-multinode-multitenant/85f60e3/ara/14:14
jrolljust can't tell where within that command14:14
jrollah, we got into safe-devstack-vm-gate-wrap.sh14:14
jrollwhich needs more set -x14:15
sambettsjroll: devstack-gate uses ansible internally so the ansible task that failed is actually being run from devstack gate14:15
*** jcoufal has quit IRC14:15
sambettsand I don't think that gets logged in ara :(14:15
jrolljebus14:15
sambettsyeah is ansible running devstack-gate which runs ansible14:15
sambetts*mind splode*14:16
sambettsideally we're not supposed to be using devstack-gate anymore14:16
jrollsounds like work14:16
sambettswe're still running "legacy" jobs14:16
*** vsaienk0 has quit IRC14:16
sambettsdevstack-gate has been replaced by nice top level ansible roles14:16
TheJuliaoh, interpretter14:17
jrollyep14:17
*** mjturek has quit IRC14:17
jrollvirtualenvs14:17
TheJuliait could also be that the the pip binaries are pointing to specific python versions14:18
TheJuliaso they could still work14:19
jrollthis is the last successful task afaict https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate-wrap.sh#L51514:19
jrollyeah, it could be14:19
jrolloh!14:23
jrollwe aren't breaking anything14:23
openstackgerritJim Rollenhagen proposed openstack/ironic master: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54475014:24
jrollneed more shebang14:24
jroll"OSError: [Errno 8] Exec format error can happen if there is no shebang line at the top of the shell script and you are trying to execute the script directly" -- should have been obvios14:24
TheJuliaI was rebooting my vm to try that *crosses fingers*14:24
jrollobvious, too14:24
* jroll brings a load of empty boxes in from the truck14:25
etingofwould it make sense to have IPv6 support in snmp driver?14:28
TheJuliaI've not looked at any of the snmp enabled strips that support v6....14:33
TheJuliabut surely one exists... somewhere...14:33
* etingof will file a bug not to forget about that14:35
*** khyr0n has joined #openstack-ironic14:36
jrolldoes ipv6 not just work with that driver? :(14:36
*** khyr0n has quit IRC14:36
*** khyr0n has joined #openstack-ironic14:37
khyr0nHi guys, have a little problem with tripleo ironic... basically the introspection works perfectly, but if I try to deploy the cloud, the baremetal node never gets and answer from the dhcp server14:37
TheJuliakhyr0n: are the dhcp requests making it into the neutron dhcp instance?14:38
*** threestrands_ has quit IRC14:39
TheJuliajroll: grenade has started \o/14:40
jroll:o nice14:40
*** mjturek has joined #openstack-ironic14:40
TheJuliahttp://zuul.openstack.org/stream.html?uuid=69effe6520394ed89d94fbfcc8e9add5&logfile=console.log for a spare monitor ;)14:40
*** cdearborn has joined #openstack-ironic14:42
*** cdearborn has quit IRC14:42
*** cdearborn has joined #openstack-ironic14:42
khyr0nTheJulia: yes, checking with tcpdump I can see only the request14:42
*** mjturek has quit IRC14:43
*** mjturek has joined #openstack-ironic14:44
*** mjturek has quit IRC14:44
jrollTheJulia: meh, strange errors, maybe this isn't the right approach :P14:47
*** dtantsur|brb is now known as dtantsur14:49
dtantsurmorning TheJulia! should we request an ironic-inspector queens release now?14:51
TheJuliadtantsur: If the backports all landed (I haven't looked yet this morning), I believe so14:53
openstackgerritJim Rollenhagen proposed openstack/ironic master: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54475014:54
jrollgrr14:54
* jroll saves some nodes at least14:54
dtantsurTheJulia: https://review.openstack.org/#/q/status:merged+project:openstack/ironic-inspector+branch:stable/queens14:55
TheJuliadtantsur: \o/14:56
TheJuliadtantsur: ship it! :)14:56
dtantsurTheJulia: totally :) I may need a +1 from you, I'm not sure releases from my are accepted any more :)14:57
TheJuliashould be, you were the ptl for the release that is being released14:58
TheJulia(of course, it likely depends on reviewer coffee levels)14:58
*** dpawar has joined #openstack-ironic14:58
TheJulia((or beer levels))14:58
TheJuliaand I can +1 stuff :)14:59
TheJuliajroll: new error "Failure from finger client: host and port was not specified and no sock specified"15:00
TheJulia:)15:00
dtantsurI'll ping you to double-check anyway15:00
jrollWAT15:00
jrollTheJulia: it's moving now :P15:01
*** rcernin has quit IRC15:01
*** mjturek has joined #openstack-ironic15:03
*** khyr0n_ has joined #openstack-ironic15:04
khyr0n_if I do a tcpdump to the namespace, like this15:07
khyr0n_ip netns exec qdhcp-16a9cf71-1b9f-4fc8-9cc6-16d512dac084 tcpdump -ln -i tap4b48966d-ef port 67 or port 6815:07
*** khyr0n has quit IRC15:07
khyr0n_I get this15:07
khyr0n_08:13:12.194709 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:98:ec:1a:ff:5c, length 54815:07
khyr0n_08:13:12.194889 IP 10.32.232.50.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 35115:07
khyr0n_And if I tail the log -> tail -f /var/log/messages | grep dnsmasq-dhcp15:08
khyr0n_I get this15:09
khyr0n_Feb 15 08:11:11 mxtlam01vas01hyp02ud dnsmasq-dhcp[23524]: DHCPDISCOVER(tap4b48966d-ef) 1c:98:ec:1a:ff:5c15:09
khyr0n_Feb 15 08:11:11 mxtlam01vas01hyp02ud dnsmasq-dhcp[23524]: DHCPOFFER(tap4b48966d-ef) 10.32.232.54 1c:98:ec:1a:ff:5c15:09
*** r-daneel has quit IRC15:09
*** hamzy_ has quit IRC15:10
TheJuliakhyr0n_: and the offer never makes it out the physical port to the barmetal?15:10
*** mrch has joined #openstack-ironic15:11
*** lucas-hungry is now known as lucasagomes15:18
khyr0n_TheJulia: no it never makes to the baremetal node15:24
etingofjroll, I do not think so15:24
jrolletingof: sadface15:24
etingofjroll, hey, this is easy to fix! ;)15:24
*** dpawar has quit IRC15:24
khyr0n_The Julia: in the baremetal I can't never see the DHCPOFFER15:25
*** dpawar has joined #openstack-ironic15:25
*** baha has joined #openstack-ironic15:25
TheJuliakhyr0n_: so that seems like a networking configuration issue if the same offer is making it all the way to the dhcp server, but not back out. Have you checked all of the firewall rules?15:26
*** rbartal has quit IRC15:27
*** etingof has quit IRC15:29
*** dpawar has quit IRC15:30
khyr0n_TheJulia: even if the introspection works perfectly? that gives me the idea that every is ok on the firewall15:31
*** hamzy_ has joined #openstack-ironic15:37
TheJuliainspection, I believe, is hooked into the networking differently15:38
*** baha has quit IRC15:38
*** baha has joined #openstack-ironic15:39
*** hamzy_ has quit IRC15:41
khyr0n_I believe that to... let me check the firewalls rules, a lot of rules exists but I'll try to accept all15:44
*** olivierbourdon38 has quit IRC15:44
TheJuliakhyr0n_: Yeah, somewhere in your controller packets are getting dropped. Most likely some sort of neutron issue, or how the networks are plugged in. Sadly the only thing one can really do at that point is walk the entire path of the packet and hunt down where it is getting dropped on the floor :(15:49
*** hamzy has joined #openstack-ironic15:50
*** crushil has joined #openstack-ironic15:57
*** r-daneel has joined #openstack-ironic15:58
NobodyCamGood Morning Ironic'ers16:00
*** mjturek has quit IRC16:01
openstackgerritMallikarjuna Reddy Kolagatla proposed openstack/ironic master: Updates boot mode on the baremetal as per `boot_mode`  https://review.openstack.org/54079316:05
TheJuliaGood morning NobodyCam16:07
NobodyCam:) Good Morning TheJulia :)16:07
*** mjturek has joined #openstack-ironic16:08
dtantsurmorning NobodyCam16:08
NobodyCamhey hey dtantsur Morning :)16:08
NobodyCamhappy almost Friday :)16:08
*** pcaruana has quit IRC16:09
*** vsaienk0 has joined #openstack-ironic16:11
dtantsuralmost happy almost Friday16:12
*** marios has quit IRC16:13
khyr0n_TheJulia: thanks, I'll try to hunt down the packages, hope to solve this problem :(16:15
khyr0n_soon :$16:17
NobodyCam:)16:17
*** mjturek has quit IRC16:17
*** ijw has joined #openstack-ironic16:18
*** khyr0n_ has quit IRC16:19
*** khyr0n has joined #openstack-ironic16:19
*** dpawar has joined #openstack-ironic16:20
*** vsaienk0 has quit IRC16:21
*** olivierbourdon38 has joined #openstack-ironic16:21
*** links has quit IRC16:22
openstackgerritMerged openstack/bifrost master: Add way to customize the IPA version to be used  https://review.openstack.org/54287316:23
*** khyr0n has quit IRC16:24
TheJuliajroll: failed :( Waiting for the logs to power... oh... I bet I know why....16:24
TheJuliahttps://review.openstack.org/#/c/544743/16:25
patchbotpatch 544743 - ironic - Revert "Clean up RPC versions and database migrati...16:25
TheJuliadtantsur: Regular grenade does not use an rpc pin, where as the pull-up job does16:25
dtantsurTheJulia: do you know when the grenade is ready for queens? it seems like most projects have stable/queens already16:26
dtantsurI'd prefer we avoid back-and-forth with revert of that patch, if possible16:27
TheJuliajust asked in infra16:27
TheJulialikewise16:27
TheJuliahmm... seems like we might see it landed soon-ish16:32
*** jcoufal has joined #openstack-ironic16:34
olivierbourdon38finally I get the error at https://github.com/openstack/ironic/blob/master/ironic/conductor/manager.py#L3232 with exception OSError: [Errno 18] Invalid cross-device link16:35
*** mjturek has joined #openstack-ironic16:35
olivierbourdon38and indeed it tries to (hard)link /var/lib/ironic/master_images/843dcd1a-459f-5610-b801-b3f9dca72256 to /httpboot/776a332a-3781-4585-96a1-712fbbe97566/ramdisk16:35
*** jcoufal_ has quit IRC16:35
olivierbourdon38any reason why this has to be a hard link ?16:35
olivierbourdon38May be I'll also add a comment in the doc to prevent people from running into these kinds of very cryptic errors (at least at first sight)16:36
*** jcoufal_ has joined #openstack-ironic16:36
*** mjturek has quit IRC16:37
dtantsurTheJulia: approved!16:37
*** e0ne has quit IRC16:37
*** AlexeyAbashkin has quit IRC16:37
olivierbourdon38I'm sure I am hitting freshmen's issues which are unfortunately taking a lot of your cycles16:38
*** mjturek has joined #openstack-ironic16:38
*** jcoufal has quit IRC16:38
openstackgerritJulia Kreger proposed openstack/ironic master: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54475016:40
mgoddard_olivierbourdon: are you using ironic in a container, or on a system with multiple filesystems mounted?16:43
mgoddard_olivierbourdon38: ^16:44
olivierbourdon38just a(nother) dummy question on this issue, wouldn't it be possible to have /httpboot folder in /var/lib instead of / ? From my perspective it seems "quick and easy" nut may be this will break some other parts which I am very unaware of as of today. Thx for your inputs16:44
*** mjturek has quit IRC16:44
sambettsthe location of the httpboot directory isn't defined by ironic, its a configuration setting16:45
*** mrch has quit IRC16:45
olivierbourdon38mgoddard_ on a system with separate partitions for / /var and /home16:45
olivierbourdon38therefore the error16:45
sambettsbifrost might be opinionated through16:45
sambettsthough*16:45
olivierbourdon38sambetts this is what I guessed from reading the bifrost code and I'll give a try to the change and test it but wanted to get experts inputs too16:46
mgoddard_olivierbourdon38: yeah, I hit this with bifrost on kolla-ansible, with these locations mounted in different docker volumes. I ended up doing this: https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/bifrost/tasks/bootstrap.yml#L4116:46
mgoddard_olivierbourdon38: that comment also references the relevant ironic bug, https://bugs.launchpad.net/ironic/+bug/150789416:47
openstackLaunchpad bug 1507894 in Ironic "Validate that images root and its master cache are on the same device" [Medium,Triaged]16:47
*** khyr0n has joined #openstack-ironic16:47
*** khyr0n has quit IRC16:47
TheJuliafun....16:48
*** khyr0n has joined #openstack-ironic16:48
olivierbourdon38may be again I am thinking way to quickly here but my idea was to use a symlink from /httpboot to <whatever dir path you want to configure in /var> because /var can grow wild by design but / is not supposed to ....16:49
olivierbourdon38but indeed your solution works great too16:49
olivierbourdon38thx16:49
sambettsI think putting the cache inside the httpboot directory is how we deploy it with devstack too16:50
openstackgerritMerged openstack/ironic master: Remove test_contains_current_release_entry  https://review.openstack.org/54273016:52
TheJuliathat would greatly avoid some possible issues16:53
TheJuliaIs anyone picking up the ansible docs patch?16:53
*** olivierbourdon38 has quit IRC16:56
*** trown is now known as trown|lunch16:56
*** baha has quit IRC17:01
pas-hameee17:03
pas-hato much comments.. :)17:03
TheJuliapas-ha: okay :)17:09
openstackgerritRyan proposed openstack/ironic master: Allow sqalchemy filtering by id and uuid  https://review.openstack.org/54504117:09
rybridgeslocasagomes: just fixed a 3 year old bug of yours https://review.openstack.org/#/c/545041/17:11
patchbotpatch 545041 - ironic - Allow sqalchemy filtering by id and uuid17:11
rybridgesWe talked about this bug in this channel like ~1.5 years ago idk if you remember17:11
rybridgesi finally got around to pushing the change upstream17:11
*** vsaienk0 has joined #openstack-ironic17:11
jrolllucasagomes: ^ heh17:12
jrollthat's a fun one17:12
jrollrybridges: you have time to add a quick unit test there?17:13
lucasagomesrybridges, jroll lol lemme take a look17:13
pas-harybridges: also, which version of sqlalchemy has this fixed? are we already requiring it? (as a dependency for oslo.db version that is in global requirements)17:14
lucasagomesrybridges, haha good catch!17:15
rybridges:)17:15
rybridgesi might have already written a unit test.. let me take a look17:16
dtantsurrybridges: cool! ideally a release note as well please17:18
* sambetts --> home 17:18
sambettsnight all17:18
*** sambetts is now known as sambetts|afk17:19
*** harlowja has joined #openstack-ironic17:19
*** mjturek has joined #openstack-ironic17:20
*** baha has joined #openstack-ironic17:20
jrollpas-ha: I don't think this depends on a certain sqlalchemy?17:20
*** ijw has quit IRC17:20
*** pcaruana has joined #openstack-ironic17:20
*** mjturek has quit IRC17:20
*** baha1 has joined #openstack-ironic17:20
dtantsurrybridges: wow, does it mean that our timeouts actually never worked O_o17:21
rybridgesyes17:21
rybridgesthat whole patch17:21
rybridgesthat i linked in my review17:21
*** vsaienk0 has quit IRC17:21
rybridgesis completely broken17:21
rybridgesi rewrote it for us internally at Oath a while back17:21
dtantsurdaaaaamnn.. and we used the same thing for more timeouts later17:22
rybridgesredid all of the unit tests to make them more robust17:22
rybridgesand in the midst of making the unit tests better17:22
rybridgesi found this problem17:22
dtantsurrybridges: this is a great catch, thanks. I'm surprised nobody noticed it before, but I did hear complains about our timeouts.17:22
rybridgesya np man17:23
dtantsurrybridges: do you have a launchpad bug for it?17:23
*** baha has quit IRC17:24
jrollamazing17:25
dtantsurfunny data point: I was about to look into one issue downstream that is probably fixed by this patch :)17:25
jrolldtantsur: which means you owe rybridges a drink, yeah? :P17:25
rybridgesi never made a launchpad bug17:25
dtantsurit seems that I do :)17:26
rybridgeslol no i already owe dtansur a million drinks for all the times he has helped me and answered my question17:26
rybridgesnow maybe we are slightly more even17:26
dtantsur:)17:26
*** dpawar has quit IRC17:28
dtantsurrybridges: mind filing a bug for tracking? just copy-pasting your commit message is fine17:28
dtantsuror jroll ^^^17:31
rybridgesdtantsur: https://bugs.launchpad.net/ironic/+bug/174975517:31
openstackLaunchpad bug 1749755 in Ironic "Allow sqalchemy filtering by id and uuid" [Undecided,New]17:31
dtantsurrybridges: thanks! when updating the patch, please link to it as Closes-Bug: #174975517:32
jrollnice, thanks17:32
jrollrybridges: let me know if you need me to pick up the test and such17:32
openstackgerritRyan proposed openstack/ironic master: Allow sqalchemy filtering by id and uuid  https://review.openstack.org/54504117:36
*** jcoufal__ has joined #openstack-ironic17:43
*** jcoufal_ has quit IRC17:43
*** dsneddon_ has joined #openstack-ironic17:48
*** mjura has quit IRC17:48
*** mjturek has joined #openstack-ironic17:49
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Fix rare HTTP 400 from port list API  https://review.openstack.org/54506917:50
dtantsurTheJulia, sambetts|afk, first take ^^^ will test/finish tomorrow17:50
dtantsurearly feedback is welcome17:51
*** dsneddon_ has quit IRC17:51
*** dsneddon_ has joined #openstack-ironic17:51
*** tesseract has quit IRC17:52
TheJuliadtantsur: okay, thanks!17:52
* TheJulia watches grenade run and crosses her fingers17:58
*** derekh has quit IRC17:59
*** AlexeyAbashkin has joined #openstack-ironic18:01
* dtantsur crosses his fingers too, while slowly leaving his work place18:02
jrollthat seemed... quick18:03
jrolloh I lied18:03
* jroll thought 'verify post-upgrade' was the tempest part18:03
*** mgoddard_ has quit IRC18:04
jrollI see some nova tests passing \o/18:04
dtantsurexciting :)18:04
*** dtantsur is now known as dtantsur|afk18:04
dtantsur|afksee you tomorrow18:04
jrollnight dtantsur|afk18:05
*** AlexeyAbashkin has quit IRC18:06
TheJuliasigh, testserveractionsjson failed18:06
TheJuliawell, at least we will hopefully see something interesting in logs...18:08
openstackgerritRyan proposed openstack/ironic master: Allow sqalchemy filtering by id and uuid  https://review.openstack.org/54504118:08
rybridgesjroll: added a releasenote. also incorporated that for loop and started raising an exception if we get an unsupported filter. hopefully i did the release note thing correctly18:09
* jroll will look18:10
jrollthanks!18:10
jrollTheJulia: test_add_remove_fixed_ip passed, which wouldn't happen if nova-conductor was busted :)18:10
TheJuliaoO18:11
TheJuliayour right18:11
TheJuliatestbasicserverops failed at 11 seconds in18:12
TheJuliaMakes me think something else is broken, but that the conductor may be partially working18:13
TheJuliaregardless, logs()18:14
jrollyep18:14
TheJulia4 errors, the behavior does look different18:18
*** baha1 is now known as baha18:22
TheJuliait _looks_ like n-cpu lost its brain during ironic's restart and was never restarted18:26
TheJuliawell, never recovered18:26
openstackgerritMerged openstack/ironic master: Empty commit to bump minor pre-detected version  https://review.openstack.org/54273118:27
jrollkinda yeah18:27
jrollit suddenly had problems talking to placement18:27
jrollidk why that would be true (unless of course we've broken queens ironic with pike nova)18:27
TheJulia17:55 is when the services began being shutdown for the upgrade18:30
TheJulia17:58 is when it gave up trying to talk to ironic18:30
*** trown|lunch is now known as trown18:30
khyr0nHi guys it's ok when you define a dhcp-range, that the first ip gets assigned to dnsmasq?... in my conf dhcp_start=10.32.232.50 but if I nmap that ip answer a dhcp server18:32
TheJulia18:00:28 is the literal last ironicclient message...18:33
TheJuliakhyr0n: sadly no, it has to be a dedicated block of unused addresses18:34
*** khyr0n_ has joined #openstack-ironic18:35
TheJulia18:00:30 is when the http proxy on the primary node got its first request and shipped it over to the subnode for ironic-api18:36
*** khyr0n_ has quit IRC18:37
*** khyr0n_ has joined #openstack-ironic18:37
*** khyr0n has quit IRC18:37
jrollTheJulia: it must continue using iroincclient after 18:00:28, just not logging it. see "using cache for node" all over afterwards, with a small age18:37
TheJuliait definitely is18:38
TheJuliaconfirmed via apache logs18:38
jrollcool18:39
*** persia has quit IRC18:39
*** mgoddard_ has joined #openstack-ironic18:39
jrollso yeah, this looks like a legit bug somehow with pike nova + queens ironic18:39
*** jcoufal__ has quit IRC18:40
*** jcoufal_ has joined #openstack-ironic18:40
*** khyr0n_ is now known as khyr0n18:43
*** jcoufal has joined #openstack-ironic18:45
*** jcoufal_ has quit IRC18:47
*** shekarvajjula has quit IRC18:48
*** shekar has joined #openstack-ironic18:49
*** ElCoyote_ has joined #openstack-ironic18:51
openstackgerritJulia Kreger proposed openstack/ironic stable/queens: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54508918:55
*** lucasagomes is now known as lucas-afk19:00
*** harlowja has quit IRC19:00
*** etingof has joined #openstack-ironic19:00
*** jcoufal_ has joined #openstack-ironic19:15
*** dprince has quit IRC19:15
*** jcoufal has quit IRC19:16
*** etingof has quit IRC19:20
*** r-daneel has quit IRC19:27
*** etingof has joined #openstack-ironic19:30
*** jcoufal_ has quit IRC19:32
*** jcoufal has joined #openstack-ironic19:33
*** harlowja has joined #openstack-ironic19:35
*** harlowja_ has joined #openstack-ironic19:37
*** harlowja has quit IRC19:39
*** dprince has joined #openstack-ironic19:40
*** dprince_ has joined #openstack-ironic19:40
*** dprince_ has quit IRC19:40
*** mgoddard_ has quit IRC19:45
*** pcaruana has quit IRC20:03
TheJulialooks like the stable/queens test is also blowing up.. different behavior it seems20:13
*** AlexeyAbashkin has joined #openstack-ironic20:15
*** AlexeyAbashkin has quit IRC20:19
*** mjturek_ has joined #openstack-ironic20:20
*** e0ne has joined #openstack-ironic20:22
*** mjturek has quit IRC20:23
*** r-daneel has joined #openstack-ironic20:31
TheJuliahahahaha /usr/bin/python2 /usr/local/bin/nova-conductor --config-file /etc/nova/nova.conf20:32
*** mjturek_ has quit IRC20:32
jrollwat!20:32
*** mjturek has joined #openstack-ironic20:32
TheJuliabecause that is pike->queens20:33
TheJuliaso in pike, we launched $stuff differently if memory serves20:33
*** jtomasek has quit IRC20:34
jrolloh, pike->queens20:35
jrollha. ha ha ha.20:35
* jroll has a masochistic moment and wonders to himself if there's a unifi/vyatta driver for neutron + baremetal out there20:36
TheJuliaHere or before is our offending pip package that <3 http://logs.openstack.org/89/545089/1/check/ironic-grenade-dsvm-multinode-multitenant/ff3fc18/logs/grenade.sh.txt.gz#_2018-02-15_20_00_12_06620:38
*** e0ne has quit IRC20:38
*** e0ne has joined #openstack-ironic20:38
*** mjturek has quit IRC20:39
TheJuliajroll: wouldn't be hard to hack into place with networking-generic-switch20:39
jrollTheJulia: yeah, that's my thought20:39
* jroll is ordering networking gear for the new house today20:39
TheJuliait is actually super-easy to do once you wrap your head around it20:40
TheJuliatroubleshooting is the major pain20:40
jrollright20:40
TheJuliabut it is really strait forward20:40
jrollI helped a bit with a similar thing for cisco/nexus at rackspace, before n-g-s happened20:40
openstackgerritJulia Kreger proposed openstack/ironic stable/queens: DNM Test disabling .pyc files for Grenade  https://review.openstack.org/54508920:44
*** mgoddard_ has joined #openstack-ironic20:46
TheJuliaahh, yeah, then you'll likely quickly find where to make more logging occur20:50
openstackgerritMark Goddard proposed openstack/ironic master: Decode traits in instance_info in API from JSON  https://review.openstack.org/54511820:59
*** crushil has quit IRC21:01
*** etingof has quit IRC21:01
*** mgoddard_ has quit IRC21:05
*** ijw has joined #openstack-ironic21:05
* TheJulia raises an eyebrow21:05
TheJuliaI'm kind of expecting the pike->queens upgrade to also bomb with disabling saving bytecode to disk, but a girl can dream that it will work flawlessly :)21:11
*** dprince has quit IRC21:13
*** etingof has joined #openstack-ironic21:14
jrolloh nice, 3rd party CI should be back now https://review.openstack.org/#/c/545033/21:25
patchbotpatch 545033 - requirements (stable/queens) - Bump u-c for tinyrpc (MERGED)21:25
jrollthanks pas-ha \o/21:25
*** e0ne has quit IRC21:31
*** pcaruana has joined #openstack-ironic21:31
*** jcoufal has quit IRC21:31
TheJuliawoot21:33
*** trown is now known as trown|outtypewww21:35
*** openstack has joined #openstack-ironic21:45
*** ChanServ sets mode: +o openstack21:45
TheJuliayes, that is a much better idea21:45
rlooTheJulia: yup. except when they fall :-(21:46
*** hamzy has quit IRC21:46
rlooTheJulia: ciao then, talk to you next week! have a good weekend!21:46
*** rloo has quit IRC21:46
*** pcaruana has quit IRC21:48
*** bfournie has quit IRC21:48
*** rcernin has joined #openstack-ironic21:50
*** baha has quit IRC21:58
*** priteau has quit IRC22:29
*** priteau has joined #openstack-ironic22:29
*** bfournie has joined #openstack-ironic22:32
*** priteau has quit IRC22:34
*** threestrands has joined #openstack-ironic22:43
*** threestrands has quit IRC22:43
*** threestrands has joined #openstack-ironic22:43
*** etingof has quit IRC22:44
*** cdearborn has quit IRC22:45
*** pmannidi has joined #openstack-ironic22:47
openstackgerritJulia Kreger proposed openstack/ironic master: DNM: Attempt to collect more info re grenade failures  https://review.openstack.org/53549222:47
openstackgerritJulia Kreger proposed openstack/ironic stable/queens: DNM: Attempt to collect more info re grenade failures  https://review.openstack.org/54514722:50
TheJuliajroll: /usr/bin/real-python2 /usr/local/bin/nova-conductor --config-file /etc/nova/nova.conf22:52
jrollTheJulia: yay!22:53
TheJuliait saw through the trap :(22:53
jrollhm? that didn't work?22:53
TheJulianope22:54
TheJuliait determined the original $0 value, and ran that somewhere along the way22:55
TheJuliaso, I think the dnm merge above should give us enough information to figure out the offending python package that is breaking stuff22:55
jrollhmm ok22:56
jrollI'm heading out, have a good night22:59
*** crushil has joined #openstack-ironic22:59
TheJuliayou too23:00
*** etingof has joined #openstack-ironic23:00
*** crushil_ has joined #openstack-ironic23:00
*** crushil__ has joined #openstack-ironic23:03
*** crushil has quit IRC23:04
*** crushil_ has quit IRC23:06
*** Goneri has quit IRC23:08
*** ijw has quit IRC23:09
*** ijw has joined #openstack-ironic23:14
*** charcol-v2 has joined #openstack-ironic23:18
*** r-daneel has quit IRC23:24
*** hshiina has joined #openstack-ironic23:36
*** hshiina2 has joined #openstack-ironic23:38
*** hshiina has quit IRC23:40
*** hshiina2 is now known as hshiina23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!