| *** rcernin has quit IRC | 00:07 | |
| *** rcernin has joined #openstack-ironic | 00:09 | |
| *** etingof has quit IRC | 00:42 | |
| *** sthussey has quit IRC | 00:51 | |
| *** jesusaur has joined #openstack-ironic | 01:12 | |
| *** tiendc has joined #openstack-ironic | 01:20 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP ipv6 job https://review.openstack.org/613169 | 01:35 |
|---|---|---|
| *** dustinc has quit IRC | 01:40 | |
| openstackgerrit | inspurericzhang proposed openstack/ironic-tempest-plugin master: Update hacking version to latest https://review.openstack.org/629347 | 02:14 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP ipv6 job https://review.openstack.org/613169 | 02:48 |
| *** rloo has quit IRC | 02:51 | |
| *** Qianbiao has joined #openstack-ironic | 03:00 | |
| *** gyee has quit IRC | 03:10 | |
| openstackgerrit | QianBiao Ng proposed openstack/ironic-specs master: Add spec for Huawei iBMC driver https://review.openstack.org/628724 | 03:35 |
| openstackgerrit | Merged openstack/sushy-tools master: Add configuration file to sushy-emulator https://review.openstack.org/616178 | 03:48 |
| *** dustinc has joined #openstack-ironic | 04:11 | |
| *** dustinc has quit IRC | 04:22 | |
| *** dustinc has joined #openstack-ironic | 04:24 | |
| *** dustinc has quit IRC | 04:35 | |
| *** dustinc has joined #openstack-ironic | 04:36 | |
| *** dustinc has quit IRC | 04:43 | |
| *** arne_wiebalck_ has joined #openstack-ironic | 04:53 | |
| *** hwoarang has joined #openstack-ironic | 04:56 | |
| *** lifeless_ has joined #openstack-ironic | 04:57 | |
| *** baha has joined #openstack-ironic | 05:01 | |
| *** dnuka has joined #openstack-ironic | 05:02 | |
| *** jesusaur has quit IRC | 05:02 | |
| *** hwoarang_ has quit IRC | 05:02 | |
| *** irclogbot_1 has quit IRC | 05:02 | |
| *** trungnv has quit IRC | 05:02 | |
| *** arne_wiebalck has quit IRC | 05:02 | |
| *** chason has quit IRC | 05:02 | |
| *** lifeless has quit IRC | 05:02 | |
| *** arne_wiebalck_ is now known as arne_wiebalck | 05:02 | |
| dnuka | good morning o/ | 05:02 |
| dnuka | good morning arne_wiebalck :) | 05:02 |
| *** irclogbot_1 has joined #openstack-ironic | 05:04 | |
| *** jesusaur has joined #openstack-ironic | 05:09 | |
| *** dustinc has joined #openstack-ironic | 05:20 | |
| *** dustinc has quit IRC | 05:29 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP ipv6 job https://review.openstack.org/613169 | 05:34 |
| openstackgerrit | Kaifeng Wang proposed openstack/ironic master: Expose conductors: api-ref https://review.openstack.org/622862 | 06:16 |
| *** e0ne has joined #openstack-ironic | 06:22 | |
| *** e0ne has quit IRC | 06:23 | |
| *** pcaruana has joined #openstack-ironic | 07:06 | |
| *** moshele has joined #openstack-ironic | 07:10 | |
| *** etingof has joined #openstack-ironic | 07:17 | |
| *** e0ne has joined #openstack-ironic | 07:18 | |
| *** e0ne has quit IRC | 07:22 | |
| arne_wiebalck | dnuka: good morning! | 07:23 |
| dnuka | o/ | 07:23 |
| *** rcernin has quit IRC | 07:26 | |
| *** hjensas has quit IRC | 07:31 | |
| *** e0ne has joined #openstack-ironic | 07:47 | |
| *** rpittau has joined #openstack-ironic | 07:54 | |
| rpittau | good morning ironic! o/ | 07:54 |
| dnuka | good morning rpittau :) | 07:54 |
| rpittau | hi dnuka :) | 07:54 |
| iurygregory | good morning everyone o/ | 08:35 |
| rpittau | hey iurygregory :) | 08:35 |
| dnuka | morning iurygregory | 08:35 |
| *** e0ne has quit IRC | 08:47 | |
| * iurygregory really love the TimeoutException =( | 08:51 | |
| *** S4ren has joined #openstack-ironic | 09:00 | |
| *** dougsz has joined #openstack-ironic | 09:13 | |
| *** hjensas has joined #openstack-ironic | 09:13 | |
| * etingof feels that being in hurry and running openstack do not always play well... ;) | 09:23 | |
| etingof | good morning everybody o/ | 09:23 |
| dnuka | morning etingof | 09:23 |
| etingof | dnuka, so did `tox -r` help? | 09:24 |
| iurygregory | morning etingof o/ | 09:24 |
| dnuka | etingof, yes :) | 09:24 |
| etingof | awesome | 09:24 |
| rpittau | hi etingof :) | 09:24 |
| *** moshele has quit IRC | 09:24 | |
| dnuka | folks, I cannot push changes to gerrit, "Problem running 'git remote update gerrit'" | 09:26 |
| iurygregory | can you check your .gitreview file in the repo? | 09:27 |
| etingof | is it what `git review` ends up with? | 09:27 |
| dnuka | ack | 09:27 |
| dnuka | Problem running 'git remote update gerrit' | 09:28 |
| dnuka | Fetching gerrit | 09:28 |
| dnuka | dnuka@review.openstack.org: Permission denied (publickey). | 09:28 |
| dnuka | fatal: Could not read from remote repository. | 09:28 |
| iurygregory | well you key seem to have wrong permisions O.o | 09:28 |
| dnuka | well, everything worked yestreday :) | 09:29 |
| dnuka | *yesterday | 09:29 |
| etingof | can it be that ssh key pass phrase has expired in the ssh agent? | 09:29 |
| * etingof would run `ssh -v dnuka@review.openstack.org` to see where auth fails | 09:30 | |
| dnuka | ack | 09:30 |
| dnuka | Permission denied :( | 09:31 |
| iurygregory | a workaround woulb be build a new key and add to gerrit =) | 09:32 |
| etingof | so `git config -l` should tell you the exact ssh command which git uses to access remote repo | 09:32 |
| etingof | then dnuka should make sure that that ssh command (including -p <port> perhaps) works out alone | 09:33 |
| dnuka | etingof, iurygregory got it :) | 09:33 |
| etingof | along the way ssh should tell dnuka which ssh key(s) it is trying to use and if they get rejected | 09:33 |
| dnuka | ack | 09:34 |
| *** robbbe has joined #openstack-ironic | 09:36 | |
| etingof | if dnuka suspects his local repo is somehow misconfigured, he could clone the upstream repo somewhere (e.g. in /tmp) and (if cloning works out) see if `git config -l` differs... | 09:36 |
| *** moshele has joined #openstack-ironic | 09:37 | |
| dnuka | etingof, got it | 09:37 |
| *** derekh has joined #openstack-ironic | 09:38 | |
| dnuka | etingof, cloning works! and git config -l` output differs | 09:41 |
| etingof | dnuka, the configuration is stored in .git/config so you can experiment with these files | 09:42 |
| dnuka | ack | 09:42 |
| *** zhangfei has joined #openstack-ironic | 09:43 | |
| etingof | dnuka, just make sure not to accidentally lose the work you've done but not yet pushed upstream | 09:43 |
| iurygregory | at least what is in gerrit is not lost =D | 09:43 |
| dnuka | etingof, almost forgot that :) | 09:44 |
| iurygregory | git review -d <3 | 09:44 |
| dnuka | iurygregory, yes :) | 09:44 |
| * rpittau looks at the clock, it's coffee time! | 09:48 | |
| dnuka | :) | 09:49 |
| iurygregory | coffee \o/ | 09:49 |
| * etingof can't get his clock reading in the morning unless initial coffeination is over | 09:52 | |
| dnuka | :D | 09:52 |
| *** S4ren has quit IRC | 09:53 | |
| rpittau | lol | 09:57 |
| *** dnuka is now known as dnuka|brb | 10:02 | |
| *** dtantsur|afk is now known as dtantsur | 10:10 | |
| dtantsur | morning ironic | 10:11 |
| *** iurygregory has quit IRC | 10:12 | |
| *** dnuka has joined #openstack-ironic | 10:13 | |
| dnuka | morning dtantsur | 10:16 |
| *** S4ren has joined #openstack-ironic | 10:18 | |
| *** e0ne has joined #openstack-ironic | 10:21 | |
| rpittau | hi dtantsur :) | 10:21 |
| *** iurygregory has joined #openstack-ironic | 10:26 | |
| *** stendulker has joined #openstack-ironic | 10:27 | |
| *** zhangfei has quit IRC | 10:32 | |
| openstackgerrit | Dhanuka Warusadura proposed openstack/sushy master: [WIP] Introduce ``convert_to_date`` https://review.openstack.org/625889 | 10:32 |
| openstackgerrit | Dhanuka Warusadura proposed openstack/sushy master: [WIP] Introduce ``convert_to_date`` https://review.openstack.org/625889 | 10:36 |
| *** iurygregory has quit IRC | 10:50 | |
| *** moshele has quit IRC | 10:57 | |
| *** stendulker has quit IRC | 11:11 | |
| *** iurygregory has joined #openstack-ironic | 11:16 | |
| *** dnuka has quit IRC | 11:16 | |
| *** dnuka has joined #openstack-ironic | 11:16 | |
| iurygregory | dtantsur, morning o/ | 11:23 |
| *** hjensas has quit IRC | 11:24 | |
| iurygregory | its normal a node have deploy_interface as None? | 11:26 |
| openstackgerrit | Hamdy Khader proposed openstack/python-ironicclient master: Add 'hostname' to port's local link connection https://review.openstack.org/628773 | 11:26 |
| rpittau | iurygregory, is that on ironic conf ? | 11:32 |
| iurygregory | rpittau, nope when looking at the node info | 11:32 |
| rpittau | I think it can be empty | 11:33 |
| iurygregory | well according to our guide we should set the node in maintenance but we cant lol | 11:34 |
| *** moshele has joined #openstack-ironic | 11:34 | |
| iurygregory | openstack baremetal node maintenance set d209a91d-5bea-4901-9809-a06697859494 --reason "Changing driver and/or hardware interfaces" | 11:34 |
| iurygregory | No valid host was found. Reason: No conductor service registered which supports driver pxe_ipmitool for conductor group "". (HTTP 400) | 11:34 |
| dtantsur | iurygregory: this is expected. no updates are possible with an invalid driver. | 11:35 |
| dtantsur | iurygregory: also deploy_interface=None is expected for classic drivers | 11:35 |
| dtantsur | also we may be debugging the same thing :) | 11:36 |
| iurygregory | cool | 11:36 |
| iurygregory | yep we are =P | 11:36 |
| * iurygregory not sure if you saw my in email in the thread lol | 11:36 | |
| dtantsur | it's an actual bug in ironic :( | 11:37 |
| iurygregory | =( | 11:39 |
| iurygregory | so our doc is wrong when we say that we can put the node in maintenance =( | 11:40 |
| iurygregory | 'It is not recommended to change the interfaces for active nodes. If absolutely needed, the nodes have to be put in the maintenance mode first' | 11:40 |
| dtantsur | iurygregory: no, this is correct. | 11:40 |
| dtantsur | it's not a workaround for updating broken nodes though | 11:41 |
| iurygregory | oh ok | 11:41 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Fix updating nodes with removed or broken drivers https://review.openstack.org/629447 | 11:45 |
| dtantsur | iurygregory: ^^^ | 11:45 |
| dtantsur | TheJulia, mgoddard if you're around ^^^ | 11:46 |
| iurygregory | dtantsur, cool you rock =D | 11:46 |
| mgoddard | dtantsur: I'll take a look | 11:47 |
| openstackgerrit | Hamdy Khader proposed openstack/python-ironicclient master: Add is-smartnic port attribute to port command https://review.openstack.org/629449 | 11:51 |
| openstackgerrit | Derek Higgins proposed openstack/ironic master: [WIP] Replace use of Q_USE_PROVIDERNET_FOR_PUBLIC https://review.openstack.org/621146 | 11:59 |
| *** iurygregory has quit IRC | 12:00 | |
| *** dnuka has quit IRC | 12:06 | |
| *** hjensas has joined #openstack-ironic | 12:07 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic stable/rocky: Fix updating nodes with removed or broken drivers https://review.openstack.org/629451 | 12:10 |
| *** dnuka has joined #openstack-ironic | 12:13 | |
| *** iurygregory has joined #openstack-ironic | 12:15 | |
| *** rpittau is now known as rpittau|lunch | 12:20 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Allocation API: allow picking random conductor for RPC topic https://review.openstack.org/629250 | 12:23 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Allocation API: conductor API (without HA and take over) https://review.openstack.org/628223 | 12:23 |
| *** tiendc has quit IRC | 12:27 | |
| *** e0ne has quit IRC | 12:30 | |
| *** dnuka has quit IRC | 12:37 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/networking-baremetal master: Change networking-baremetal to zuulv3/python3 https://review.openstack.org/614026 | 12:39 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-inspector master: Follow-up Zuulv3 (Ironic jobs rename) https://review.openstack.org/629475 | 13:05 |
| *** trown|outtypewww is now known as trown | 13:09 | |
| *** jcoufal has joined #openstack-ironic | 13:16 | |
| *** rpittau|lunch is now known as rpittau | 13:21 | |
| *** e0ne has joined #openstack-ironic | 13:24 | |
| *** vkmc has left #openstack-ironic | 13:29 | |
| *** Qianbiao has quit IRC | 13:51 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic master: Remove dsvm from zuulv3 jobs https://review.openstack.org/629173 | 13:54 |
| *** rloo has joined #openstack-ironic | 13:55 | |
| *** moshele has quit IRC | 13:57 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-inspector master: Follow-up Zuulv3 (Ironic jobs rename) https://review.openstack.org/629475 | 13:58 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-tempest-plugin master: Rename zuulv3 jobs https://review.openstack.org/629494 | 14:07 |
| *** baha has joined #openstack-ironic | 14:16 | |
| openstackgerrit | Ilya Etingof proposed openstack/sushy master: Add System<->Manager linkage follow up https://review.openstack.org/629500 | 14:19 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Allocation API: conductor API (without HA and take over) https://review.openstack.org/628223 | 14:26 |
| TheJulia | dtantsur: do we have an elastic recheck in place for the failed to read errors? | 14:32 |
| dtantsur | TheJulia: nope. at least not from me. I can look into it. | 14:35 |
| dtantsur | (and thanks for approving that bug fix) | 14:35 |
| TheJulia | dtantsur: thanks, I wonder just how often we're seeing it | 14:35 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/networking-generic-switch master: Rename ironic job to match zuulv3 https://review.openstack.org/629508 | 14:35 |
| iurygregory | morning TheJulia o/ | 14:36 |
| dtantsur | TheJulia: quite often I'm sure | 14:36 |
| dtantsur | I wonder if the recent IPA change is involved.. or is it just bionic | 14:36 |
| dtantsur | TheJulia: do you have an example elastic recheck update handy? | 14:36 |
| TheJulia | dtantsur: i don't sadly, but I am super curious about that change | 14:37 |
| rpittau | good morning TheJulia :) | 14:37 |
| TheJulia | I never really agreed with it, tbh | 14:37 |
| TheJulia | but I wasn't going to block it | 14:37 |
| dtantsur | why didn't you agree? | 14:38 |
| TheJulia | because we shouldn't make assumptions into the contents we write | 14:41 |
| TheJulia | How do we know what is being written has or needs a partition table? | 14:41 |
| TheJulia | shouldn't be an issue on any of the platforms really used with ironic though, for most common things | 14:41 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: [WIP] Allocation API: implementation https://review.openstack.org/629260 | 14:42 |
| dtantsur | TheJulia: hmm.. maybe you're right.. | 14:43 |
| dtantsur | TheJulia: as to elastic-recheck, I cannot even make logstash output me *anything* >_< | 14:43 |
| TheJulia | dtantsur: ugh :( | 14:43 |
| TheJulia | maybe clarkb has some magical insight there he can point us to? | 14:44 |
| TheJulia | re platform stuffs, I am quite altruistic though, not quite practical in that regard, so it is likely practical, but worth digging at that change. I can dig at that with a fresh set of eyes once coffee hits and I'm done with email | 14:45 |
| dtantsur | okay, the syntax is funny, but I seem to start figuring it out | 14:46 |
| dtantsur | TheJulia: 95 hits in a week for message:"Failed to create config drive on disk" AND message:"test -e" :( | 14:50 |
| TheJulia | EEK | 14:50 |
| TheJulia | We had a lot of failures late last week, so that could be it | 14:50 |
| TheJulia | test -e seems... a little not specific enough because I think it is body contents matching not line level matching | 14:51 |
| * TheJulia has no idea how the cluster is indexed | 14:51 | |
| dtantsur | TheJulia: well, it seems to find what we need | 14:52 |
| dtantsur | now creating a story for this | 14:52 |
| *** baha has quit IRC | 14:52 | |
| TheJulia | \o/ | 14:53 |
| TheJulia | almost done with email, so I can begin deep diving on the failure | 14:54 |
| dtantsur | TheJulia: https://storyboard.openstack.org/#!/story/2004744 | 14:54 |
| TheJulia | awesome | 14:55 |
| iurygregory | yay finally finished the patchs to rename jobs \o/ | 14:58 |
| dtantsur | TheJulia: the query shows that at least sometimes it failed on xenial, so we may be able to rule out bionic as a cause | 15:04 |
| dtantsur | TheJulia: and happens only with wholedisk+direct jobs.. which make the IPA patch an even bigger suspect | 15:05 |
| TheJulia | hmmm | 15:06 |
| dtantsur | hmm, wait, the IPA code runs for all image types | 15:06 |
| TheJulia | Well, it is clear the person with huawei is awake right now, they've responded to two emails so far this morning | 15:07 |
| *** chason_ has joined #openstack-ironic | 15:09 | |
| *** mjturek has joined #openstack-ironic | 15:10 | |
| dtantsur | TheJulia: hmm, does this actually mean that we should run partprobe (again!) http://logs.openstack.org/47/629447/1/check/ironic-tempest-dsvm-ipa-wholedisk-direct-tinyipa-multinode/65739ae/compute1/logs/ironic-bm-logs/subnode-0_no_ansi_2019-01-09-12:38:08_log.txt.gz#_2019-01-09_12_37_27_550 ? | 15:12 |
| TheJulia | That is likely what is happening, but that would only fix the symptom | 15:13 |
| * TheJulia wonders | 15:13 | |
| dtantsur | and why for love's sake are we using GPT in a BIOS job? Oo | 15:13 |
| iurygregory | magic | 15:14 |
| TheJulia | oO | 15:14 |
| dtantsur | because yes, we do: http://logs.openstack.org/47/629447/1/check/ironic-tempest-dsvm-ipa-wholedisk-direct-tinyipa-multinode/65739ae/compute1/logs/ironic-bm-logs/subnode-0_no_ansi_2019-01-09-12:38:08_log.txt.gz#_2019-01-09_12_37_21_815 | 15:14 |
| *** baha has joined #openstack-ironic | 15:14 | |
| TheJulia | dtantsur: the image we are writing ? | 15:15 |
| dtantsur | I mean, it is supported, but why are we doing it? | 15:15 |
| dtantsur | well, yes, that's the explanation | 15:15 |
| TheJulia | we recently bumped the version of the image, I suspect it is a silent change | 15:16 |
| dtantsur | we recently moved to new version of cirros, can it bring GPT here? /cc derekh | 15:16 |
| TheJulia | that is what I'm thinking | 15:16 |
| TheJulia | I'm glad we put in the gpt repair logic | 15:17 |
| dtantsur | vda: vda1 vda15 | 15:17 |
| dtantsur | WUT | 15:17 |
| dtantsur | no, srsly WUT | 15:17 |
| dtantsur | dear linux.. | 15:17 |
| derekh | dtantsur: dunno | 15:18 |
| TheJulia | wut^wutwut | 15:19 |
| iurygregory | O.o | 15:19 |
| dtantsur | I need whiskey. a lot of whiskey. | 15:20 |
| iurygregory | after see this i need to agree | 15:20 |
| TheJulia | 2 is the config drive, 15 is the /boot volume | 15:21 |
| * TheJulia suspects it is too early to open the japanese whiskey | 15:22 | |
| dtantsur | yeah, except that, I suspect, the configdrive is now always 2... | 15:22 |
| dtantsur | I have some great stuff from Islay from my birthday. But too early even for me.. | 15:22 |
| * iurygregory miss Japanese beer. | 15:22 | |
| dtantsur | another guess is that we need to insert partprobe into yet another place | 15:23 |
| dtantsur | this is a better option for us | 15:23 |
| dtantsur | we can even try going down this path first | 15:24 |
| TheJulia | looks like it would have to be a whdi to hit this failure | 15:25 |
| TheJulia | dtantsur: ++ | 15:26 |
| TheJulia | looks like we should just be able to do it in ipa which would cover that case... feels like we have another bug someplace in the path | 15:29 |
| dtantsur | who knows... I'm patching ironic-lib for now | 15:30 |
| dtantsur | TheJulia: I wonder if https://review.openstack.org/#/c/594922/ may be involved | 15:33 |
| patchbot | patch 594922 - ironic-lib - Check GPT table with sgdisk insread of partprobe (MERGED) - 2 patch sets | 15:33 |
| TheJulia | So we pulled down the partition image... | 15:33 |
| openstackgerrit | Ilya Etingof proposed openstack/sushy master: Add Chassis<->ComputerSystem/Manager linkage https://review.openstack.org/623028 | 15:33 |
| TheJulia | err no | 15:34 |
| TheJulia | yeah, wholedisk is what it was | 15:34 |
| iurygregory | "nat" is correct? | 15:35 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-lib master: [WIP] Run partprobe after adding a configdrive partition https://review.openstack.org/629547 | 15:36 |
| dtantsur | TheJulia: let's see what the CI says bout ^^^ | 15:36 |
| TheJulia | ++ | 15:37 |
| TheJulia | cirros is not starting out with a partition table :\ | 15:40 |
| iurygregory | TheJulia, would be necessary push a change to openstack-infra/project-config since the have the name for one job that is changed? http://codesearch.openstack.org/?q=ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa&i=nope&files=&repos=project-config | 15:45 |
| openstackgerrit | Ilya Etingof proposed openstack/sushy-tools master: Add memoization to expensive emulator calls https://review.openstack.org/612758 | 15:46 |
| NobodyCam | Good Morning and happy hump day | 15:50 |
| iurygregory | good morning NobodyCam o/ | 15:51 |
| dtantsur | morning NobodyCam | 15:51 |
| NobodyCam | O/ iurygregory and dtantsur :) | 15:51 |
| rpittau | hi NobodyCam :) | 15:54 |
| NobodyCam | Good Morning rpittau :) | 15:55 |
| TheJulia | something is super fishy about this failure :( and no sushy involved | 15:56 |
| TheJulia | likely, that is performance graphing someone setup at some point... | 15:57 |
| TheJulia | doh, its a qcow, not raw image | 15:59 |
| TheJulia | okay, after converting | 16:00 |
| TheJulia | yes, the 0.4.0 is a gpt filesystem | 16:00 |
| dtantsur | \o/ | 16:03 |
| dtantsur | so we've lost coverage for BIOS+MBR? :D | 16:03 |
| *** pcaruana has quit IRC | 16:03 | |
| TheJulia | for wholedisk, yes | 16:03 |
| TheJulia | but gpt is fine | 16:03 |
| TheJulia | err partition based is fine | 16:04 |
| TheJulia | we have to trust the image" and we are doing it in whole disk jobs | 16:04 |
| TheJulia | we're just much more resilliant apparently | 16:04 |
| dtantsur | yeah, but GPT and MBR are different code paths even for wholedisk images (I think) | 16:04 |
| TheJulia | dtantsur: interleaved | 16:04 |
| dtantsur | yeah. in quite some spaghetti :) | 16:05 |
| TheJulia | indeed | 16:06 |
| TheJulia | I'm actually really happy that it passed | 16:06 |
| TheJulia | well | 16:06 |
| TheJulia | gets through | 16:06 |
| TheJulia | it fails for other reasons it seems | 16:06 |
| * TheJulia gets out the colander | 16:09 | |
| * TheJulia wonders if anyone uses the partial code for local caching for nodes awaiting official deployment orders | 16:13 | |
| dtantsur | good question | 16:14 |
| *** gyee has joined #openstack-ironic | 16:18 | |
| *** moshele has joined #openstack-ironic | 16:24 | |
| TheJulia | something is really not adding up for me, maybe I just need more coffee | 16:27 |
| dtantsur | something? I'm completely in darkness with this issue :) if "just throw another partprobe" does not work, I dunno.. | 16:28 |
| TheJulia | I think that is the only option, starting to understand | 16:30 |
| *** moshele has quit IRC | 16:32 | |
| * TheJulia opens up parted's source code to confirm theory | 16:32 | |
| dtantsur | wow | 16:32 |
| *** rnoriega has quit IRC | 16:38 | |
| *** radez has quit IRC | 16:38 | |
| openstackgerrit | Merged openstack/sushy master: Add System<->Manager linkage follow up https://review.openstack.org/629500 | 16:38 |
| TheJulia | sweet... github failed | 16:39 |
| TheJulia | oh, its back now :( | 16:39 |
| *** ltomasbo has quit IRC | 16:39 | |
| *** lucasagomes has quit IRC | 16:39 | |
| TheJulia | wow, the way it is written makes my head hurt | 16:40 |
| iurygregory | dtantsur, should i trigger the check-experimental in nova since some ironic jobs run only there? (2 jobs are in experimental) https://review.openstack.org/#/c/629504/1/.zuul.yaml | 16:41 |
| patchbot | patch 629504 - nova - Rename Ironic jobs - 1 patch set | 16:41 |
| *** lucasagomes has joined #openstack-ironic | 16:41 | |
| *** rnoriega has joined #openstack-ironic | 16:43 | |
| TheJulia | yeah, I suspect my feeling is right | 16:44 |
| *** lucasagomes has quit IRC | 16:45 | |
| *** lucasagomes has joined #openstack-ironic | 16:50 | |
| dtantsur | iurygregory: yeah, you can try it | 16:51 |
| dtantsur | TheJulia: so? | 16:51 |
| iurygregory | dtantsur, ack tks o/ | 16:51 |
| TheJulia | still sifting through the c code | 16:51 |
| TheJulia | I just confirmed it should be fsyncing, so it is likely just that, the kernel is not loading the table afterwards. There is a note on the fsync docs about behavior dependant upon filesystem though... | 16:53 |
| dtantsur | "udevadm settle" returned: 0 in 38.853s <- impressive | 16:54 |
| TheJulia | wow | 16:54 |
| * TheJulia saves other words for not online | 16:54 | |
| dtantsur | I see. I'm looking at http://logs.openstack.org/47/629547/1/check/ironic-lib-wholedisk-bios-ipmi-direct-tinyipa-src-python2/f7961c8/controller/logs/ironic-bm-logs/node-0_no_ansi_2019-01-09-16:28:34_log.txt.gz#_2019-01-09_16_28_03_502 I cannot see my partprobe. weird. | 16:55 |
| TheJulia | because we don't build it in by default afaik | 16:55 |
| TheJulia | oh, did we build ipa from scratch with the code? | 16:55 |
| dtantsur | TheJulia: well, we used to build IPA in ironic-lib gate at least | 16:56 |
| dtantsur | but I have a feeling it got broken. sigh. SIGH. | 16:57 |
| TheJulia | ugh | 17:01 |
| dtantsur | oh, I see. we hardcode /opt/stack/new in some places. it's no longer the thing.. | 17:03 |
| *** e0ne has quit IRC | 17:03 | |
| * TheJulia gets out whiskey | 17:03 | |
| TheJulia | fsync on a ramdisk is an explicit noop | 17:03 |
| TheJulia | it does nothing | 17:03 |
| *** robbbe has quit IRC | 17:03 | |
| TheJulia | https://github.com/torvalds/linux/blob/6f0d349d922ba44e4348a17a78ea51b7135965b1/fs/ramfs/file-mmu.c | 17:04 |
| rpittau | I was going to ask a couple of things but I see the whiskey level is already high enough, it can wait :P | 17:04 |
| NobodyCam | rpittau: LOL | 17:04 |
| TheJulia | so io data from parted is in the memory cache, and waits upon the system to do lazy writing out since it doesn't appear to do explicit direct io calls | 17:05 |
| TheJulia | reading however for the kernel requires a lock from the device, so there is the case without forcing a disk sync that the new partition is in ram, but not yet on disk when we attempt to rescan | 17:06 |
| rpittau | bye all! good evening o/ | 17:08 |
| NobodyCam | have a good night rpittau | 17:08 |
| *** rpittau has quit IRC | 17:08 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: Recover CI run detection in imagebuild https://review.openstack.org/629574 | 17:09 |
| dtantsur | TheJulia: maybe this can help ^^^ but I'm too tired to think more | 17:09 |
| TheJulia | dtantsur: I think we need to perform an explicit sync as well to totally eliminate slow disk io being an isue | 17:09 |
| dtantsur | TheJulia: in my ironic-lib patch? | 17:10 |
| TheJulia | that will force the kernel to go disk by disk syncing out buffers | 17:10 |
| TheJulia | dtantsur: yeah, I can do it | 17:10 |
| dtantsur | yeah, feel free to update it | 17:10 |
| *** dustinc has joined #openstack-ironic | 17:10 | |
| TheJulia | k | 17:10 |
| TheJulia | coffee first | 17:10 |
| * etingof has been wondering about explicit sync | 17:10 | |
| dtantsur | see you | 17:10 |
| *** dtantsur is now known as dtantsur|afk | 17:11 | |
| TheJulia | yeah, seems silly to ahve to do it, but ramdisk the rules are different | 17:11 |
| TheJulia | goodnight | 17:11 |
| * etingof observed that sysadmins tend to run `sync` before committing into some serious business | 17:12 | |
| etingof | felt like a genetic trait | 17:12 |
| TheJulia | yeah, provides some guarantee (minus raid controllers lying about io cache flush-out) that things are written to disk | 17:16 |
| TheJulia | in case, oh... accidental power button or the like | 17:16 |
| TheJulia | I had a database platform I was responsible for the datacenter operations and system administration of a long time ago, and it used a mix of direct io and let the rest of the io be a lazy write to disk. The aspects that were a lazy write are almost always the part that led to corruption | 17:18 |
| TheJulia | and it was possible to also crash some of the raid controllers then with massive direct io queues | 17:19 |
| TheJulia | soooo many database restores.... | 17:19 |
| etingof | so the habit of occasional syncing is a good trait ;) | 17:19 |
| *** moshele has joined #openstack-ironic | 17:19 | |
| TheJulia | indeed | 17:23 |
| openstackgerrit | Merged openstack/networking-generic-switch master: Directly source ML2 plugin for devstack https://review.openstack.org/628708 | 17:28 |
| *** moshele has quit IRC | 17:28 | |
| TheJulia | Interesting dmitry's wip patch failed pep8 :\ but is fine locally | 17:31 |
| TheJulia | oh, no, it did | 17:31 |
| TheJulia | wrong line \o/ | 17:31 |
| openstackgerrit | Derek Higgins proposed openstack/ironic master: [WIP] Replace use of Q_USE_PROVIDERNET_FOR_PUBLIC https://review.openstack.org/621146 | 17:49 |
| *** dougsz has quit IRC | 17:54 | |
| *** trown is now known as trown|lunch | 17:55 | |
| *** derekh has quit IRC | 17:56 | |
| *** MattMan has quit IRC | 18:11 | |
| *** MattMan has joined #openstack-ironic | 18:11 | |
| *** dustinc has quit IRC | 18:22 | |
| *** dustinc has joined #openstack-ironic | 18:23 | |
| *** S4ren has quit IRC | 18:30 | |
| *** e0ne has joined #openstack-ironic | 18:35 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-lib master: Run sync and partprobe after adding a configdrive partition https://review.openstack.org/629547 | 18:44 |
| *** e0ne has quit IRC | 19:02 | |
| openstackgerrit | Hamdy Khader proposed openstack/ironic master: Add is_smartnic to Port data model https://review.openstack.org/629640 | 19:04 |
| *** e0ne has joined #openstack-ironic | 19:05 | |
| *** trown|lunch is now known as trown | 19:08 | |
| *** moshele has joined #openstack-ironic | 19:50 | |
| *** e0ne has quit IRC | 20:08 | |
| *** dustinc has quit IRC | 20:37 | |
| *** S4ren has joined #openstack-ironic | 20:51 | |
| *** dustinc has joined #openstack-ironic | 20:52 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP ipv6 job https://review.openstack.org/613169 | 21:12 |
| TheJulia | Core reviews on https://review.openstack.org/#/c/629574/1 would be appreciated | 21:16 |
| patchbot | patch 629574 - ironic-python-agent - Recover CI run detection in imagebuild - 1 patch set | 21:16 |
| *** moshele has quit IRC | 21:29 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: WIP ipv6 job https://review.openstack.org/613169 | 21:57 |
| *** baha has quit IRC | 22:01 | |
| *** trown is now known as trown|outtypewww | 22:10 | |
| TheJulia | and core reviews on https://review.openstack.org/#/c/629547/ would also be appreciated | 22:31 |
| patchbot | patch 629547 - ironic-lib - Run sync and partprobe after adding a configdrive ... - 2 patch sets | 22:31 |
| *** rcernin has joined #openstack-ironic | 22:52 | |
| *** jcoufal has quit IRC | 23:03 | |
| jidar | can I re-run the undercloud installer (openstack undercloud install) and move the interface over from say, eno3 to eno2 if I messed it up? | 23:22 |
| *** dustinc has quit IRC | 23:33 | |
| *** dustinc has joined #openstack-ironic | 23:36 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!