Tuesday, 2014-04-01

*** Hefeweiz1n has quit IRC00:02
*** eghobo has quit IRC00:03
*** dwalleck has quit IRC00:08
NobodyCamoff to look for dinner00:11
JayFIt seems like every so often, it just takes hours upon hours to get tests to run for things00:25
JayFis there some kind of problem? Not enough capacity?00:25
JayFWe've had a patch set queued in jenkins for 90m or so now00:25
*** matsuhashi has joined #openstack-ironic00:27
*** JayF has quit IRC00:40
*** JayF has joined #openstack-ironic00:41
NobodyCamlol tests keep getting longer and longer :)00:43
*** datajerk has joined #openstack-ironic00:49
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Add BackOffLoopingCall with jitter  https://review.openstack.org/8430300:58
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Add BackOffLoopingCall with jitter  https://review.openstack.org/8430300:59
*** zigo has quit IRC00:59
*** zigo has joined #openstack-ironic01:01
devanandaJayF: http://status.openstack.org/zuul/01:08
*** nosnos has joined #openstack-ironic01:26
*** dwalleck has joined #openstack-ironic01:28
NobodyCamdevananda: do you know of a good how to use the .po files doc you can point me to?01:36
devanandanope01:36
devanandaactualy01:36
* devananda goes to retest it01:36
NobodyCamhttps://wiki.openstack.org/wiki/I18NSupport01:36
NobodyCamfound that one01:37
NobodyCam:-p01:37
*** dwalleck has quit IRC01:40
devanandaurgh01:45
devanandastill not working01:45
devanandaNobodyCam: wow, that's really old01:46
*** dwalleck has joined #openstack-ironic01:48
Shrewsancient, even01:53
devanandaso01:53
Shrewsla01:54
devanandathe manual import of i18n'd strings is not, actually, enough to get translation working for me01:54
devanandaand combined with haomeng's patch, still not doing it01:54
devanandaor I'm doing it wrong01:54
devanandabut, for example I can issue01:54
devanandacurl -X GET -H 'Accept-Language: pt_BR' -H 'Content-Type: application/json' http://127.0.0.1:6385/v1/nodes/?limit=-1001:54
openstackgerritA change was merged to openstack/ironic: Manually update all translated strings  https://review.openstack.org/8425901:54
devanandawhich should be returning the translated message string01:55
devanandamsgstr "Limite deve ser positivo"01:55
devanandabut isn't01:55
Shrewsugh, already with the april fool's stuff: http://blog.freenode.net/2014/04/googleplusfreenode/02:01
devanandamordred: hi! around? got a sec?02:05
lifelessdevananda: CI is back up02:30
devanandalifeless: \o/02:31
devanandalifeless: so. should we be treating that job as voting?02:31
lifelessdevananda: e.g. http://status.openstack.org/zuul/ see 8404302:32
lifelessdevananda: I know of no reason for it to fail other than code bugs [somewhere :P]02:32
devanandaheh. great02:32
lifelessdevananda: the undercloud job we don't know if it works yet, but we'll have more data when that one finishes02:33
devanandalifeless: random ? - you have knowledge of openstack's i18n / gettext hooks? I'm trying to figure out why it's not working right ...02:33
lifelesssorry no02:33
lifelessI haven't looked at that yet02:33
Haomeng|2devananda: let me try to test my patch with non-en language, and show you the result02:36
devanandaHaomeng|2: hi there!02:36
Haomeng|2yes:)02:37
devanandaHaomeng|2: i have been testing your patch with current master (the i18n strings landed just a bit ago) and have not been able to get it to work02:37
devanandaHaomeng|2: please let me know how it goes. I'll be around for a while02:37
Haomeng|2devananda: my patch have two fixes, one is to enable  lazy translation to follow oslo change, another one is we do the formatting in common exception class to make sure the error message is in correct charset and encode02:38
Haomeng|2devananda: ok, I will take other project's  lazy translation enablement as reference02:39
Haomeng|2devananda: I think I need some time to investigate why it is not working, so dont wait me for long time if you want to go to bed:)02:42
devanandaHaomeng|2: I have a lot still to do tonight... will be up anyway for several hours more02:43
Haomeng|2devananda: ok02:43
Haomeng|2Haomeng|2: devananda: and I think maybe our ironic i18n not working  without the  lazy translation enablement, let me try to verify, now facing some env issue02:44
Haomeng|2Haomeng|2: ok02:45
Haomeng|2devananda: did you change po files to enable non-en strings, by default, all language string are in english I think02:54
devanandaHaomeng|2: hm?02:54
devanandaHaomeng|2: what do you mean, "enable non-en strings"?02:54
Haomeng|2devananda: let me try to enable non-en strings in po files and test it, means to set non-en string for non-en po files, let me do it, and let you know the result:)02:55
devanandaHaomeng|2: there are now many non-en strings, eg, http://git.openstack.org/cgit/openstack/ironic/tree/ironic/locale/zh_CN/LC_MESSAGES/ironic.po02:59
Haomeng|2devananda: ok, let me fix my env and test them out02:59
*** harlowja is now known as harlowja_away02:59
*** dwalleck has quit IRC03:02
*** killer_prince has quit IRC03:08
*** harlowja_away is now known as harlowja03:09
*** rameshg87 has joined #openstack-ironic03:10
*** radsy has quit IRC03:12
*** matsuhashi has quit IRC03:22
devanandaHaomeng|2: if I change ironic/openstack/common/gettextutils.py: gettext.translation(... fallback=True) to False03:28
devanandaHaomeng|2: then I get: IOError: [Errno 2] No translation file found for domain: 'ironic'03:28
Haomeng|2devananda: "gettextutils.install('ironic', lazy=True)" this code is suggestted by oslo, and I take othere project as reference, they have same code change03:29
devanandaHaomeng|2: yes. I think that is a valid change. there is another problem too03:29
Haomeng|2Haomeng|2: my env is back, let me continue to test this03:29
devanandai just dont know what the other problem is, yet03:29
Haomeng|2devananda: yes03:29
devanandaI am experimenting with manually setting03:30
devanandaexport IRONIC_LOCALEDIR=/opt/stack/ironic/03:30
Haomeng|2devananda: let me do some error message and to see if it can be display with non-en language03:30
devanandaHaomeng|2: even starting the conductor service should produce non-en messages in the log now03:30
devanandathat is how i'm testing03:30
Haomeng|2devananda: yes03:31
devanandaby making gettext not 'fallback' to en, it is erroring out quickly03:31
Haomeng|2devananda: do you remove this line 'gettextutils.enable_lazy()' to rollback the code?03:32
devanandano03:33
Haomeng|2devananda: I think both L18, L19 in this file https://review.openstack.org/#/c/74626/10/ironic/cmd/__init__.py enable the lazy mode to support 'Accept-Language' request I think03:35
devanandaHaomeng|2: i have changed LC_ALL in the host03:35
devanandaHaomeng|2: so translation is being done for log messages as well03:35
Haomeng|2devananda: great03:35
devanandait makes testing faster :)03:35
Haomeng|2devananda: good idea:)03:36
Haomeng|2devananda: let me continue to test the api call with Accept-Language03:36
Haomeng|2devananda: if log message is translated, that mean our code is working03:36
*** nosnos has quit IRC03:37
devanandaHaomeng|2: well. it attempts to translate log message, but fails to find locale_dir and falls back to english03:38
Haomeng|2devananda: this should be an issue I think03:40
devananda?03:40
Haomeng|2devananda: but I think oslo gettext code should get the relativepath in our ironic/locale03:41
devanandai have overridden local dir to /opt/stack/ironic/ironic/locale03:41
devanandabut still get03:41
devanandaIOError: [Errno 2] No translation file found for domain: 'ironic'03:41
devanandaeven though files exist in /opt/stack/ironic/ironic/locale/zh_CN/LC_MESSAGES/ironic.po03:42
Haomeng|2did we remove this line "gettextutils.install('ironic', lazy=True)" which is registing our ironic domain03:42
devanandano. should I ?03:43
devanandaahh03:43
devanandaHaomeng|2: so if I remove both of the lines you added to cmd/__init__03:43
devanandathen ironic-conductor starts03:44
devanandabut does not translate anything03:44
devanandaso your patch definitely causes ir-cond to *attempt* to translate03:44
Haomeng|2devananda: the orig code has only one line "    gettextutils.install('ironic')", which has not enable lazy i18n03:44
*** eghobo has joined #openstack-ironic03:44
devanandaright. this does not work.03:45
devanandaI modified the oslo gettextutils code to add some logging03:45
devanandaif I revert your patch, that logging is not triggered03:45
Haomeng|2devananda: strange03:46
devanandaI am learning a lot about the translation code03:46
devanandabut it seems odd that lazy=True is required for log messages when the system LC_CLANGUAGE is already set03:46
Haomeng|2devananda: I understand "lazy=True" is just for api which support on-demand  with the input language03:48
devanandaright03:49
devanandaexcept it looks like we need that for the LOG messages too ....03:49
devanandaoh. one sec03:49
Haomeng|2devananda: log message language is depending on the current  /etc/sysconfig/i18n language configuration I think03:49
devanandamay be an artefact of where i added logging :)03:49
Haomeng|2devananda: :)03:50
devanandaHaomeng|2: ye! that was it. I can revert your patch and get translation to try to run. it still fails to find the localedir though :)03:51
devanandaHaomeng|2: so you did not intend to enable lazy translation for the conductor?03:53
Haomeng|2devananda: the lazy enablement is on globel level I think for both conductor and api, it is in our ironic/cmd/__init__.py code03:54
devanandacorrect03:54
devanandabut that seems unnecessary03:54
devanandashouldn't lazy only be enabled for the api?03:55
devanandaahh. nvm03:55
devanandaahh03:56
devanandahmmmm03:56
Haomeng|2devananda: yes, if lazy is used for api only, it is not required for conductor03:56
Haomeng|2Haomeng|2: :)03:56
Haomeng|2devananda: let me check other project code change03:56
devanandaHaomeng|2: if it should only be lazy in the api,then your change should be applied to ironic/cmd/api.py and ironic/api/app.wsgi03:58
Haomeng|2devananda: yes, let me have try04:00
Haomeng|2devananda: looks like other project are enableed lazy mode on globel level - https://github.com/openstack/glance/blob/master/glance/cmd/__init__.py#L1704:00
Haomeng|2devananda: not sure if they have good testing as well:)04:01
Haomeng|2devananda: my env still have problem...04:02
Haomeng|2devananda: will get back for a while...04:06
*** lazy_prince has joined #openstack-ironic04:11
*** lazy_prince is now known as killer_prince04:11
*** matsuhashi has joined #openstack-ironic04:14
*** nosnos has joined #openstack-ironic04:23
adam_g*sad face*04:25
adam_gdevananda,  i guess this is what we'll need to do till tempest provides a better way http://paste.ubuntu.com/7188082/04:26
devanandaadam_g: heh. it's not as bad as I expected ....04:27
devanandaadam_g: admittedly i'm not that familiar with tempest's test suite yet. so maybe this is everything and i just dont realize it04:28
adam_gstill pretty fugly04:28
devanandai'd almost prefer one line per category04:28
adam_gdevananda, whadya mean?04:29
devanandabut i dont know how infra would feel04:29
devanandaadam_g: # skip tests that do <thing> ; r="$r|..."04:29
devanandapretty much just split up L1304:30
adam_gdevananda, oh, yeah unfortunately tempest.api.compute.* ends up stressing other non-nova stuff04:30
devanandaah04:30
adam_gill see if i can clean it up some more before submitting it tomorrow04:31
adam_grunning it against devstack now to confirm we at least get all green04:31
devanandacompute.test_images_one_server sounds like the thing we'd want to run04:31
devanandabut i haven't looked at the actual tests04:32
adam_gdevananda, it creates an image from a running server, ie snapshot04:32
devanandaooh04:32
devanandaheh04:32
adam_g:)04:32
devanandaright. skip it :)04:32
adam_gwhile you're around... how do i get https://review.openstack.org/#/c/84043/ to depend on a more recent patchset in 84033?04:33
*** killer_prince2 has joined #openstack-ironic04:35
devanandaadam_g: start with the latest 84033, cherry-pick 05d3b3ee3, git review -R04:35
adam_gdevananda, ah thanks04:36
devananda-R will avoid potentially rebasing 84033, so even though git review will prompt you "are you sure you want to upload multiple thinsg" it actually wont04:36
devanandaHaomeng|2: I'm going to toss up a slight change to your patch, in another patchset04:41
Haomeng|2devananda: that is fine for me04:41
devanandaHaomeng|2: i also noticed that your commit has a poor author line -- Author: root <whaom@cn.ibm.com>04:41
devanandaHaomeng|2: i dont think your name is root :)04:41
Haomeng|2devananda: :)04:42
Haomeng|2devananda: that is default os user name:)04:42
devanandaHaomeng|2: git config --global user.name XXX04:42
Haomeng|2devananda: let me fix the name, yes04:42
devananda:)04:42
Haomeng|2devananda: thanks for your reminder:)04:42
openstackgerritAdam Gandelman proposed a change to openstack/ironic: Update Ironic Nova drivers' unit tests  https://review.openstack.org/8404304:43
Haomeng|2devananda: I have checked our team guys which focus on i18n, he think the lazy mode does not effect the log message, so why othere project set it on globel I think04:44
Haomeng|2devananda: but we can just enable lazy mode for api only04:45
*** sabah has joined #openstack-ironic04:47
*** subah has joined #openstack-ironic04:57
*** subah has quit IRC04:58
*** zul has quit IRC05:00
*** matsuhas_ has joined #openstack-ironic05:02
*** matsuhas_ has quit IRC05:02
devanandaHaomeng|2: ok. so even with that fix, api is not attempting to translate strings05:03
devanandaHaomeng|2: right now we can only get logged translations, not api translations05:03
devanandaHaomeng|2: we are missing a translation hook in the api middleware05:03
*** zul has joined #openstack-ironic05:04
*** matsuhashi has quit IRC05:05
Haomeng|2devananda: but i understand the _("") hook is installed by our code     gettextutils.install('ironic') on global level05:05
devanandayes05:05
devanandait's not quite that simple though05:05
Haomeng|2devananda: yes05:05
*** matsuhashi has joined #openstack-ironic05:05
devanandathe api service isn't changing the requested language05:05
devanandawhen calling Message.translate, the desired_locale defaults to None.05:06
devanandathis needs to be passed in by the API05:06
devanandaor it will default to the system locale05:06
Haomeng|2devananda: ok, we can call "    gettextutils.install('ironic') " in our api code to have try05:07
devanandawhich was already working05:07
lifelessdevananda: hey so05:08
Haomeng|2devananda: yes, we have to change   /etc/sysconfig/i18n to set non-en language05:08
lifelessironic is failing on deploys05:08
devanandalifeless: eh?05:08
lifelessdevananda: in CI; I'm filing a bug now - but AFAICT its no long an infrastructure issue05:08
lifelesslonger that is05:08
lifelessdevananda: https://bugs.launchpad.net/ironic/+bug/130058905:10
*** harlowja is now known as harlowja_away05:11
lifelessdevananda: note that the seed doesn't actually *deploy* anything - if we add a baremetal cirros test or something like that then we'd be able to show this failure in the seed job05:12
lifelessdevananda: [thats a known thing-to-fix]05:12
*** killer_prince has quit IRC05:14
devanandalifeless: ERROR Timeout reached when waiting callback for node05:17
devanandalifeless: so. why did it time out?05:17
lifelessdevananda: I don't know05:17
devanandalifeless: this is suspicious: ERROR Request returned failure status05:17
devanandafrom glance05:17
lifeless2014-04-01 03:31:17.878 3865 WARNING keystoneclient.middleware.auth_token [-] Authorization failed for token05:18
lifeless2014-04-01 03:31:17.878 3865 WARNING keystoneclient.middleware.auth_token [-] Authorization failed for token05:18
devanandalifeless: in the ir-cond log?05:18
lifelessdevananda: its odd that you'd get an error from glance and then still timeout rather than erroring immediately :) that might point to a bug05:18
lifelessdevananda: thats glance log05:18
lifelessdevananda: the logs are in the seed-logs.tgz05:18
devanandaso that ought to have failed when ir-cond first pulled the images down during prepare()05:19
lifelesstimestamps may be important here05:19
lifelessthe 31:17 lines up exactly05:19
devanandaBuffer I/O error on device loop0p105:20
devanandaany concern there?05:20
lifelessanything after 02:55 is when we have started the deploy (so are assuming the seed is fully configured, images loaded etc)05:20
lifelessdib triggers that loop0p1 fairly routinely05:20
lifelesswe've not tracked any negatives to it at this point- believed to be a udev race (udev reading a device we've removed)05:21
Haomeng|2devananda: this is heat code change - https://review.openstack.org/#/c/69133/5/heat/common/wsgi.py which call gettextutils.translate in wsgi05:21
Haomeng|2devananda: maybe we need such call in our ironic wsgi, let me find05:21
devanandalifeless: provisioning starts at05:23
devananda[01/Apr/2014 02:55:50] "PUT /v1/nodes/f36bc755-df36-40fc-8d57-9504e194eadd/states/provision05:23
devanandalifeless: there's no issue with glance auth at that time05:23
devanandalifeless: so what changed between 2:55 and the glance auth failure?05:24
lifelessdevananda: nothing - last line in os-collect-config is05:24
lifeless[2014-04-01 02:40:46,791] (os-refresh-config) [INFO] Completed phase migration05:24
lifelessdevananda: so all our deploy glue was idle05:25
openstackgerritlokesh s proposed a change to openstack/python-ironicclient: adding ironic command line interfaces  https://review.openstack.org/8434605:25
lifelesskeystone log :05:25
lifeless2014-04-01 02:40:48.286 3794 WARNING keystone.common.wsgi [-] Could not find user, ironic.05:25
lifeless2014-04-01 03:31:17.876 3794 WARNING keystone.common.wsgi [-] Could not find token, admin.05:25
lifelessthat 2:40 one is a bit odd05:25
*** loki_ has joined #openstack-ironic05:25
Haomeng|2devananda: maybe we need a customized wsgi to enable the exception translation05:30
lifelessdevananda: could possibly be just that ironic used an expired token05:33
devanandalifeless: are you guys expiring tokens quickly?05:33
lifelessdevananda: defaults :)05:34
devanandalifeless: which is?05:34
lifelessas much as possible05:34
lifelesselements/keystone/os-apply-config/etc/keystone/keystone.conf05:34
lifelessI believe the default is one hour05:34
devananda8640005:35
devanandais what's commented out in that keystone.conf05:35
lifelessyeah05:35
devanandawhich is far longer than 22 minutes05:35
lifelesswhich would have come from keystone.conf.sample05:35
lifelesswhere are you getting 22 minutes from ?05:35
devanandabad math05:36
devananda02:40 - 03:3105:36
devananda51 minutes, not 2205:36
devanandai was looking at a different number05:37
devanandastill should not have caused the token to expire05:37
lifeless2:55 is when the deploy started05:37
lifelesscould you have gotten the token before the deploy started?05:37
lifeless(when does Ironic acquire tokens? and does Ironic automatically retry when a token failure is encountered ?)05:38
*** sabah has quit IRC05:41
devanandahmm05:46
devanandait's conceivable that clean_up happened with a context that wasn't auth'd for glance access05:47
devanandalifeless: so it's not an issue with keystone token05:47
devanandalifeless: both prepare and clean_up call _get_tftp_image_info, which calls out to glance using our glance_service wrapper05:48
devanandathey both pass in the existing context, which may include an auth token05:48
devanandabut if clean_up is called from a periodic task (which it was in this case)05:48
devanandathen taht token was generated by ironic, not by nova (or nova's user)05:49
devanandaso05:49
devanandathat's potentially a bug05:49
devanandabut not the cause of your problem05:49
lifelessclean_up is called from nova delete?05:49
lifelessdevananda: so - this should be reproducible05:50
lifelessdevananda: pushing up patches should let you gather more info in the same way as with e.g. devstack-gate05:50
devanandaclean_up may be called from 'nova delete' or from ironic.conductor.manager05:50
devananda_check_deploy_timeouts05:51
lifelessdevananda: if I deploy two instances with the same root image, and a single conductor, will it be shared between them ?05:51
devanandai believe so, yes05:51
lifelesscool05:51
lifelessso, I did some basic timing with Ironic the other day05:52
lifelessand it was slower than our nova-baremetal runs05:52
lifelesswhich surprised me a little05:52
lifelessbut I haven't dug in05:52
lifelessso I need to context switch05:52
lifelessif you need anything admin privileged to help debug this please let me know05:53
devanandalifeless: afaict the VM did not DHCP boot05:53
devanandalifeless: which is why it eventually timed out05:53
lifelesswe're running nobodycams patch for ci_commands05:53
lifelessor it would have errored on power management much earlier and not tried the deploy05:53
devanandai'm not sure how that relates to it not DHCPBOOTing05:54
*** pradipta_away is now known as pradipta05:54
lifelessdevananda: the VM configuration is static05:54
devananda...05:54
lifelessdevananda: and identical for all of them; only way it coud not DHCP boot is if it wasn't powered on05:54
devanandahow can it get tftp targets then?05:54
devanandai dont see any requsets to tftpd in the logs05:55
devananda(or what ever tftpd was replaced with)05:55
devanandaor is there a static tftpd elsewhere too?05:55
lifelessApr  1 03:00:38 ubuntu dnsmasq-dhcp[4833]: DHCPDISCOVER(tap1e5579ec-96) 52:54:00:b5:63:b705:56
lifelessApr  1 03:00:38 ubuntu dnsmasq-dhcp[4833]: DHCPOFFER(tap1e5579ec-96) 192.0.2.3 52:54:00:b5:63:b705:56
lifelessso - thats DHCP boot05:56
lifelessof the VM05:56
devanandathat MAC is not the MAC that was enrolled with ironic05:56
devanandaer. nvm05:56
devanandamaybe it was05:56
* devananda wishes POST content was logged somewhere ...05:56
lifelesshttps://jenkins03.openstack.org/job/check-tripleo-ironic-undercloud-precise/123/console is doing the same thing05:57
lifelessso I don't think its a one-off virsh fail05:57
lifelesshttp://logs.openstack.org/43/84043/2/check-tripleo/check-tripleo-ironic-undercloud-precise/a956ec9/console.html has the full testenv descriptor at the top05:57
lifeless52:54:00:b5:63:b7 is the MAC of the first node in the nodes list05:57
lifelessso that is indeed the VM registerer with Ironic05:58
devanandaok.05:58
lifelessif it wasn't that MAC dnsmasq would not have responded05:58
devanandaso. should we be seeing a log from tftp when the VM requested the kernel/ramdisk/token?05:58
devanandaI do within my devstack env05:58
lifelesswhere does that get logged?05:59
devanandavar/log/syslog05:59
lifelessok so possible failures06:01
lifelesswe know BM1 -> seed worked for DHCP06:01
lifelesswe've no reason to expect TFTP to fail06:01
lifelessTFTPD might not be running06:01
lifelessthe port might not have PXE options configured on it06:03
lifelesse.g. if you signal neutron to set pxe06:03
lifelessand then boot the vm before neutron has hupped the dnsmasq process06:04
lifelessyup06:05
lifelesshuge race condition06:05
devanandalifeless: in theory, it wouldn't get the dhcp response that we see in the logs if that were the case06:05
devanandabut you're using a static dhcp config06:05
devanandai dunno how that will play with tftpd06:05
devanandaafaik, that shouldn't affect tftp. ironic has prepared the tfptd files before we power on the machine06:06
lifelessdevananda: static dhcp config ?06:06
lifelessdevananda: no you misunderstand06:06
devananda?06:07
openstackgerritJenkins proposed a change to openstack/ironic: Imported Translations from Transifex  https://review.openstack.org/8395606:07
lifelessdevananda: neutron is doing the dhcp right06:07
lifelessdevananda: _update_neutron is called to pass the needed PXE boot options into neutron06:07
lifelessdevananda: and then without waiting we power on the VM06:09
lifelessdevananda: see comment 1 in the bug06:09
lifelessdevananda: the call to neutron returns when neutron has the request in the DB06:09
lifelessdevananda: not when the dnsmasq agent is updated06:09
devanandaright06:10
devanandabut if that were the problem here06:10
devanandawe'd not see the DHCP RESPOSNE06:10
devanandabut we do06:10
*** loki__ has joined #openstack-ironic06:10
lifelessdevananda: yes you would06:10
devananda?06:10
lifelessdevananda: because the port when created can do DHCP06:10
lifelessdevananda: just not PXE06:10
devanandaah06:10
devanandaurgh. right.06:10
devanandait's late :)06:11
lifelessyes06:11
lifelessso I'm going to add a 10 seconds delay there and push a branch06:11
lifelessand we'll see if it passes06:11
devanandak06:11
*** loki_ has quit IRC06:11
devanandai'm going to spin off the glance issue into a new bug06:11
devanandathen sign off for the night06:12
Haomeng|2devananda: one question, do you know how to add transations for our ironic wsgi api call? we need it to enable the lazy translations, maybe we have to do it in next release06:14
Haomeng|2devananda: we have no such wsgi.py code to customize the api call06:15
devanandalifeless: fwiw, https://bugs.launchpad.net/ironic/+bug/130060206:15
Haomeng|2devananda: we are different with heat wsgi code I think06:16
devanandaHaomeng|2: yes. two places06:16
lifelessdevananda: thanks06:16
openstackgerritlifeless proposed a change to openstack/ironic: Workaround race with Neutron port updates  https://review.openstack.org/8436106:16
Haomeng|2devananda: ok, I want to enable it if possible06:16
Haomeng|2devananda: is this one ParsableErrorMiddleware?06:17
devanandaHaomeng|2: yep. so take a look at ceilometer's ParsableErrorMiddleware06:17
devanandaours is similar, but lacks the translation hook06:17
Haomeng|2devananda: ok06:17
Haomeng|2devananda: thank you06:17
Haomeng|2devananda: have a good dream:)06:17
devanandaHaomeng|2: to enable lazy in the api, you need to add it to both06:17
Haomeng|2devananda: yes06:17
devanandacmd/api.py and api/app.wsgi06:18
Haomeng|2devananda: ok, got:)06:18
Haomeng|2devananda: thanks06:18
devanandaapi.py is for python WSGI app. api/app.wsgi is for Apache mod_wsgi applications06:18
devanandaHaomeng|2: thanks! I made some progress and can post it, but it's not working yet06:18
Haomeng|2devananda: ok, got06:19
Haomeng|2devananda: my pleasure:)06:19
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Add lazy translation to API tier  https://review.openstack.org/8436206:20
devanandaHaomeng|2: feel free to take what you can from ^ and fit it into your patch.06:20
Haomeng|2devananda: ok, let me see your code06:20
Haomeng|2devananda: sure, thank you:)06:21
devanandathank you :)06:21
devanandagood night06:21
Haomeng|2devananda: :)06:21
*** saju_m has joined #openstack-ironic06:21
Haomeng|2devananda: I am in afternoon, good night for you, and have a nice dream:)06:21
openstackgerritlokesh s proposed a change to openstack/python-ironicclient: adding ironic command line interfaces  https://review.openstack.org/8434606:33
*** nosnos has quit IRC06:45
*** nosnos has joined #openstack-ironic06:46
*** matsuhashi has quit IRC06:58
*** matsuhashi has joined #openstack-ironic06:58
*** jistr has joined #openstack-ironic07:02
*** matsuhashi has quit IRC07:03
*** matsuhashi has joined #openstack-ironic07:06
GheRiveromorning all07:15
Haomeng|2GheRivero: morning:)07:15
dtantsurMorning Ironic07:20
openstackgerritMichael Davies proposed a change to openstack/ironic: Handling validation in conductor consistently  https://review.openstack.org/8437407:24
*** mrda is now known as mrda_away07:29
*** ndipanov has joined #openstack-ironic07:33
*** eghobo has quit IRC07:45
openstackgerritSergey Nikitin proposed a change to openstack/ironic: Improved coverage for ironic API  https://review.openstack.org/8437607:48
*** max_lobur has joined #openstack-ironic07:53
*** athomas has joined #openstack-ironic07:54
*** killer_prince has joined #openstack-ironic08:02
*** matsuhashi has quit IRC08:03
*** matsuhas_ has joined #openstack-ironic08:05
*** lucasagomes has joined #openstack-ironic08:13
*** romcheg has joined #openstack-ironic08:21
*** killer_prince has quit IRC08:23
*** romcheg has quit IRC08:23
*** derekh has joined #openstack-ironic08:24
*** romcheg has joined #openstack-ironic08:24
*** matsuhas_ has quit IRC08:33
*** matsuhashi has joined #openstack-ironic08:38
*** saju_m has quit IRC08:43
yuriyzmorning Ironic08:46
Haomeng|2yuriyz: morning:)08:46
romchegMorning guys08:53
*** saju_m has joined #openstack-ironic08:56
*** athomas has quit IRC08:59
*** yonglihe_ has quit IRC09:00
lucasagomesmorning yuriyz Haomeng|2 romcheg09:04
Haomeng|2lucasagomes: morning:)09:04
romcheglucasagomes: My US visa was approved. So I will go to the summit, if nothing special happens :)09:05
lucasagomesromcheg, w00t!!! nice!09:05
Haomeng|2romcheg: congts:)09:06
*** overlayer has joined #openstack-ironic09:07
*** pradipta is now known as pradipta_away09:19
*** martyntaylor has quit IRC09:26
*** martyntaylor has joined #openstack-ironic09:26
*** jistr has quit IRC09:39
*** athomas has joined #openstack-ironic09:50
*** athomas has quit IRC09:56
*** athomas has joined #openstack-ironic09:59
*** jistr has joined #openstack-ironic09:59
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Replace sfdisk with parted  https://review.openstack.org/8439610:12
*** matsuhashi has quit IRC10:14
rameshg87hello10:21
rameshg87when does ironic open for juno ? we were a few items which were moved from icehouse and was asked to resubmit when juno reopens10:22
lucasagomesrameshg87, hi, I don't know exactly :/ but I'm expecting it to open this week10:23
*** ndipanov has quit IRC10:23
lucasagomesI think that after Ironic get's tagged as RC110:23
lucasagomeswe are good to go10:23
rameshg87thanks :-)10:24
*** overlayer has quit IRC10:26
*** overlayer has joined #openstack-ironic10:31
*** overlayer has quit IRC10:34
*** ndipanov has joined #openstack-ironic10:36
*** nosnos has quit IRC10:40
*** overlayer has joined #openstack-ironic10:40
*** overlayer has quit IRC10:42
*** overlayer has joined #openstack-ironic10:45
*** overlayer has quit IRC10:59
*** rameshg87 has left #openstack-ironic11:06
*** mdurnosvistov1 has joined #openstack-ironic11:14
*** sphoorti has joined #openstack-ironic11:29
*** lucasagomes is now known as lucas-hungry11:34
*** loki__ has quit IRC11:39
*** saju_m has quit IRC11:44
openstackgerritJenkins proposed a change to openstack/ironic: Updated from global requirements  https://review.openstack.org/8347111:50
*** saju_m has joined #openstack-ironic11:56
*** saju_m has quit IRC12:02
romcheglucas-hungry: Your last patch is intended to replace 83399 and 83396, isn't it?12:04
romchegI like KISS methodology :)12:05
lucas-hungryromcheg, for the rc1 release only, 83399 and 83396 are plumbings for other stuff (like getting rid of the swap partition, GPT partitions, layout consistency)12:05
lucas-hungrybut it's too big to land for the rc12:05
lucas-hungryso the new one is just a short-term fix12:06
romchegAh, I see12:06
romchegI thought we planned to get rid of low level operations like disk partitioning in Ironic and move them to the agent?12:06
romcheg*in the next release*12:07
lucas-hungryyeah, the agent it another driver12:07
lucas-hungrythe default pxe will still need some partitioning logic I think12:07
lucas-hungryis another driver*12:07
romchegMakes sense12:07
romchegI regret missing the meet up :(12:07
lucas-hungry:) no worries12:08
lucas-hungryyeah in the logs there's a bunch of talks related to the disk stuff12:08
*** overlayer has joined #openstack-ironic12:10
*** romcheg has left #openstack-ironic12:10
*** romcheg has joined #openstack-ironic12:10
lucas-hungrybrb lunch is done :)12:11
romcheglucas-hungry: Bon appetite!12:11
*** ndipanov has quit IRC12:18
*** overlayer has quit IRC12:18
*** saju_m has joined #openstack-ironic12:21
*** saju_m has quit IRC12:23
*** ndipanov has joined #openstack-ironic12:30
*** rloo has joined #openstack-ironic12:34
*** lucas-hungry is now known as lucasagomes12:36
*** saju_m has joined #openstack-ironic12:39
*** linggao has joined #openstack-ironic12:39
*** jdob has joined #openstack-ironic12:40
*** jbjohnso_ has joined #openstack-ironic12:41
*** mtrisoline has quit IRC12:42
*** jgrimm has joined #openstack-ironic12:43
*** yuriyz has quit IRC12:47
*** overlayer has joined #openstack-ironic12:47
*** yuriyz has joined #openstack-ironic12:50
*** overlayer has quit IRC12:54
*** jbjohnso_ has quit IRC12:58
*** killer_prince2 has quit IRC12:58
*** jbjohnso_ has joined #openstack-ironic13:13
*** saju_m has quit IRC13:15
dtantsurAny ideas what to do with error "RPC do_node_tear_down not allowed for node e8d9b219-4c7f-44b5-baa5-00d939b35180 in state None"?13:16
dtantsurThis happened after unsuccessful deploy because my nodes were in maintenance mode (for unknown reason)13:17
dtantsurNow I cannot even nova delete this instance13:17
jrolldtantsur: I've had the same problem, I ended up dropping to a mysql shell on ironic to delete the instance, then nova can delete it ok13:19
jroll(if I remember correctly)13:19
jrollalso, good morning ironic!13:19
dtantsurjroll, morning, thanks. So there should be something wrong in error handling path of either nova or ironic...13:20
jrolldtantsur: yeah13:22
jrollthis is where I left off friday, and I didn't take a look yesterday13:22
jrollbut will be working on it today or tomorrow sometime, I think13:23
dtantsurjroll, do we have bug # for it?13:23
openstackgerritJim Rollenhagen proposed a change to openstack/ironic-python-agent: Add kernel parameter support  https://review.openstack.org/8426413:24
rloodtantsur: my guess if your node went to maintenance mode, is that the periodic task to sync the power state didn't work and after 3 times was put in maint mode :-(13:24
jrolldtantsur: not yet, I haven't investigated enough that I'm sure it's a bug13:24
dtantsurjroll, not being able to delete failed instance is definitely a bug :)13:25
rloodtantsur: would be good if you would open a bug about it13:25
jrolltouché :P13:25
dtantsurrloo, jroll ok, I'll open a bug13:25
jrollawesome, thanks13:26
rloothx dtantsur!13:26
jrolldtantsur: my rationale may have been that it was 6:30 pm on a friday and I wasn't sure if I was doing something dumb :P13:26
jrollagordeev: quick review ftw :)13:27
*** saju_m has joined #openstack-ironic13:28
lucasagomesdtantsur, jroll I opnened a bug about it, I'm taking a look at it right now13:28
lucasagomeshttps://bugs.launchpad.net/ironic/+bug/130028813:28
jrollah ha!13:29
dtantsurlucasagomes, cool!13:29
jrollI was wondering where teardown was being called, I was looking for some deploy failed case13:29
lucasagomesbefore we didn't see that cause the validation of the do_node_deploy() was async13:29
lucasagomesjroll, it's nova13:29
jrollthanks lucasagomes :)13:29
jrollahhh13:29
lucasagomesrescheduling the instance13:30
jrolllucasagomes: right, saw that in the bug :)13:30
Haomeng|2hello every one, who know how to generate mo file for our ironic po files to enable i18n?13:30
dtantsurnow I have one more problem: after I start boot, I see Ironic probing 3 nodes in a row (they are virtual, so I see the console), tftp succedes, kernel boots, then machine shuts down, next one is selected...13:33
dtantsurAfter all, boot fails with  NoValidHost13:33
dtantsurAn ideas, what the cause is?13:34
*** jgrimm has quit IRC13:34
Haomeng|2have to run  "python msgfmt.py *.mo" to create?13:35
rlooHaomeng|2, what's a .mo file? (ie, I don't know.)13:36
NobodyCamgood morning Ironic...13:38
romchegMorning NobodyCam13:38
NobodyCamhey hey romcheg :)13:38
rloomornin' NobodyCam13:38
NobodyCammorning rloo :)13:39
agordeevmorning NobodyCam, also morning/evening Ironic!13:39
NobodyCamand mornign agordeev :)13:39
NobodyCammorning even :-p13:39
agordeevrloo: it's all about translation strings. I saw devanda had been asking the same question in openstack-dev last night13:40
rlooagordeev: do you know if he got an answer? maybe Haomeng|2 should look there13:41
agordeevah, devananda, it meant13:41
agordeevrloo: nope, he asked clarkb, but he didn't know too13:41
rlooHaomeng|2: ^^13:42
agordeevHaomeng|2: this is somehow related to babel. Are you familiar with it?13:43
rloolucasagomes: question about docn for python-ironicclient. do you have any thoughts about it?13:46
rloolucasagomes: I see keystone has a README, and wiki.13:47
rloolucasagomes: https://github.com/openstack/python-keystoneclient/blob/master/README.rst?#id1013:47
rloolucasagomes: http://docs.openstack.org/user-guide/content/keystoneclient_commands.html13:47
*** jbjohnso_ has quit IRC13:49
NobodyCamwow I was just reading the story book that is our last night scroll back13:50
NobodyCamhave not finished13:50
NobodyCamhave we tested the translation stuff?13:51
romchegShame on me. I didn't do the Ukrainian translation.13:58
lucasagomesrloo, hmm, idk really, before I thought about a man for the cli13:59
lucasagomesbut I had some push backs on that14:00
* lucasagomes trying to remember the reasons14:00
rloolucasagomes: I remember that. I think it was because we thought it would be too much to maintain?14:00
lucasagomesthe keystone readme looks good14:00
lucasagomesrloo, I think so14:00
rloolucasagomes: I just saw this review: https://review.openstack.org/#/c/84346/2/doc/source/ironic-commands.rst14:01
lucasagomeshttps://review.openstack.org/#/c/52902/14:01
*** jbjohnso_ has joined #openstack-ironic14:01
NobodyCamromcheg: do you know how to test the translations?14:01
romchegNobodyCam: hmm, I can judge on Ukrainian and Russian: There's no way to test translations14:02
rloolucasagomes: so do we have usages examples in the developer page?14:02
romchegYou have to be a human being to make translations14:02
NobodyCamromcheg: I've searched for how to set them up doc.. and been unable to find a good one14:03
lucasagomesrloo, hmm I don't think we have any cli examples there no14:03
romchegWe can only check the coverage14:03
romchegLike how many text was translated14:04
lucasagomesrloo, so, according to devananda comment at https://review.openstack.org/#/c/52902/14:04
romchegSpellchecking is not possible because of a lot of context-specific things14:04
lucasagomeswe should have some examples in the dev docs14:04
lucasagomeswhich I agree14:05
lucasagomesfor both, CLI and LIB14:05
*** max_lobur1 has joined #openstack-ironic14:05
rloolucasagomes: if https://review.openstack.org/#/c/84346/2 is good enough wrt the CLI commands, maybe examples can be added here?14:05
*** max_lobur has quit IRC14:07
lucasagomesrloo, I think so, I don't have a strong opinion where it should go tho, I'm fine having examples in the dev doc... But I also think that a wiki would be easier to maintain14:08
rloolucasagomes: ok, let me snoop around at the other projects and see if there is some consistency or not. That review looks like it needs a bit of work too...14:09
NobodyCamromcheg: when you say theres no way to test translations. I'm unclear. I am look to do the reverse. I would like to set up a test system to see the translated text even if I don't understand it14:10
lucasagomesrloo, heh yeah, there's some Ironic (capitalized there) which will fail :P14:10
devanandagood morning, all14:10
NobodyCamgood morning devananda :) wow late night for you lastnight14:10
rloolucasagomes: exactly. might be other stuff too, i'm not sure how much development this person has done.14:10
rloomornin' devananda14:10
devanandaNobodyCam: yea... trying to fix translation14:11
lucasagomesyeah14:11
lucasagomesmorning devananda14:11
NobodyCami have a mo file. I just dont know how to use it14:11
NobodyCam:-p14:11
devanandaoh!14:11
devanandahow'd you get that???14:11
NobodyCammsgfmt -o locale/es/LC_MESSAGES/ironic.po locale/es/LC_MESSAGES/ironic.mo14:12
NobodyCamyou may nneed to apt-get install gettext14:12
NobodyCambut what do I do with said .mo file14:12
romchegNobodyCam: It's not a problem to get a translated text, but what's the next step?14:12
NobodyCamI'll happly convert my seed to a different lang for testing14:13
*** saju_m has quit IRC14:14
devanandaheh14:14
devanandaso there are 2 situations where i18n should happen14:14
devanandaLOG messages when system language is different14:15
devanandaAPI responses when header Accept-Language is different14:15
devanandaHaomeng's patch should be addressign the API response -- it needs to be "lazy translated", iow, dynamic based on the user's request14:15
devanandaand LOG translation should already have been working14:15
devanandathere were a lot of problems14:16
devananda* our .po files contained none of the translated strings which were in transifex (FIXED)14:16
devananda* jenkins wasn't submitting the translations (pending patch in infra)14:16
devananda* openstac/common/gettextutils is looking in system default location, so ironic needs to be packaged. install from source doesn't put the langs in the right place14:17
romchegdevananda: Sometimes we format error messages which then go both to the log and to the API response.14:17
devananda(can work around with export IRONIC_LOCALEDIR=/path/to/locale)14:17
devanandathere's also no logging when gettextutils fails to find a translation14:18
lucasagomesNobodyCam, hey there, when you get a time can u give it a go and see if it's working fine in ur devtest env? https://review.openstack.org/#/c/84396/14:18
devanandait silently falls back to en_US14:18
romchegdevananda: If system language is different from the Accept-Language, that will cause a problem14:18
lucasagomesI tested here in devstack, looks fine14:18
devanandaromcheg: yes14:18
devanandaromcheg: so this is something we will need to address in Juno14:19
romchegI also think this is sort of out of Ironic's scope14:19
NobodyCamlucasagomes: I tested the pated patches yesterday. they worked then.. have they changed14:20
*** jgrimm has joined #openstack-ironic14:20
romchegBecause the other projects also use localisation14:20
lucasagomesNobodyCam, right... yeah it does pretty much the same... that's the short-term fix for icehouse that we disucssed yesterday in the meeting14:21
NobodyCamack14:21
NobodyCamgive me a couple of min14:21
devanandaromcheg: it's the way we are creating messages in conductor and passing back to API14:21
devanandaromcheg: where we pass an exception it should work14:21
lucasagomesNobodyCam, np! thanks14:21
devanandaromcheg: api can localize the exc14:22
devanandaromcheg: but it can't localize what we save to node.last_error, for example -- that has to be done by conductor (if it should be done at all, which is questionable)14:22
romchegdevananda: Yes, that is the fix for this problem. But testing that is quite hard.14:22
NobodyCambrb14:23
dtantsurlucasagomes, back to https://bugs.launchpad.net/ironic/+bug/1300288 will it also fix inability to delete failed instances?14:23
*** mdurnosvistov1 has quit IRC14:23
romchegAnd it requires more projects than just Ironic14:24
romchegs/requires/is required/14:24
*** lazy_prince has joined #openstack-ironic14:27
*** lazy_prince is now known as killer_prince14:27
lucasagomesdtantsur, yes14:30
lucasagomesdtantsur, well at least I expect it to fix that too14:31
lucasagomesdtantsur, I've a quick hacky fixing it and it allows me to delete it afterwards14:31
*** killer_prince has quit IRC14:34
*** lazy_prince has joined #openstack-ironic14:34
*** lazy_prince is now known as killer_prince14:35
NobodyCamlucasagomes: testing now14:36
lucasagomesNobodyCam, :D cheers14:37
dtantsurlucasagomes, any ideas on this: http://paste.openstack.org/show/74756/ ? is it partition-flavor size mismatch or something else?14:41
*** ciranttech has joined #openstack-ironic14:41
devanandaNobodyCam: that cmd does not seem to create valid .mo files14:43
devanandaIOError: [Errno 0] Bad magic number: '/opt/stack/ironic/ironic/locale/zh_CN/LC_MESSAGES/ironic.mo'14:43
devanandaNobodyCam: in fact, it doesn't compile the .mo at all14:43
lucasagomesdtantsur, hmmm haven't seem that... bad input/no partitions found... this whole disk stuff is quite broke right now14:43
lucasagomesthe size of the disk supports the size of the partitions?14:44
NobodyCam:( devananda finding info on this is tuff.. I'll jump back on it right after I test lucasagomes is patch14:44
NobodyCamit at least created a .mo file for me14:44
devanandawe already have .po files14:44
devanandathis is creating that format14:44
dtantsurlucasagomes, not sure. My "baremetals" are just libvirt VMs with 21G hard drive14:44
lucasagomesdtantsur, sounds like enough space for tests heh14:45
lucasagomesdtantsur, so, hmm you mind trying it with the new patch I put up?14:45
dtantsurlucasagomes, which one exactly?14:45
lucasagomesdtantsur, https://review.openstack.org/#/c/84396/14:46
dtantsurwill try14:46
NobodyCammsgfmt -o locale/es/LC_MESSAGES/ironic.mo locale/es/LC_MESSAGES/ironic.po14:46
NobodyCamhahhah I had the order wrong in my paste14:46
NobodyCamsorry :(14:46
lucasagomesdtantsur, the sfdisk + 1MB lead space is causing a lot of misaligned problems, I hope that ^ will fix it14:46
*** ciranttech has left #openstack-ironic14:47
lucasagomesI mean that ^ 8439614:47
devanandagot it!14:50
NobodyCamdevananda: the mo file?14:50
devanandaNobodyCam: Haomeng|2: pybabel compile -i <file.po> -o <file.mo>14:50
devanandaor14:51
dtantsurlucasagomes, Command: sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s /dev/disk/by-path/ip-192.0.2.14:3260-iscsi-iqn-c1361bc2-b28e-461f-a5a7-1714cf2c2d4c-lun-1 -- mklabel msdos unit MiB mkpart primary  0 21504 mkpart primary linux-swap 21504 2150514:51
devanandacd ironic/locale && for file in $(ls -1 */LC_MESSAGES/ironic.po); do pybabel compile -i $file -o ${file%%.po}.mo ; done14:51
dtantsurlucasagomes, 'Error: The location 21504 is outside of the device /dev/sda.\n'14:51
lucasagomesdtantsur, a-ha!  so the disk is too small14:51
dtantsurlucasagomes, too small for what?14:51
devanandaawesome. now when I change LC_ALL, conductor logs are translated14:51
lucasagomesdtantsur, the root partition14:52
lucasagomescheck the properties local_gb14:52
lucasagomes21504 = 21 G?14:52
dtantsurlucasagomes, u'local_gb': u'21'14:52
lucasagomesrigth yeah14:52
lucasagomesand what's the size of the disk of the machine ur trying to deploy to?14:54
dtantsurlucasagomes,  mkpart primary linux-swap 21504 21505   <-- is it correct?14:54
NobodyCamdevananda: the command I used did create a mo file.. I orginaly pasted the command with the options reversed14:54
dtantsurlucasagomes, virt-manager shows 21.00 GB14:54
lucasagomesdtantsur, unfortunately... in trunk, if no swap is specified we create a partition with 1MB14:54
lucasagomesdtantsur, which sucks I know14:54
lucasagomesdtantsur, https://review.openstack.org/#/c/83726/ fix that14:55
lucasagomesbut it won't land until juno opens14:55
dtantsurlucasagomes, I think we try to create partition (N, N+1), maybe we should try (N-1, N) instead? N - size of disk14:55
lucasagomesdtantsur, +114:55
lucasagomesdtantsur, actually we need to detect it before14:55
devanandaNobodyCam: it's not "just a .mo file" -- it's a specific format of file14:55
lucasagomesdtantsur, https://review.openstack.org/#/c/78001/14:56
devanandaNobodyCam: msgfmt doesn't create them, babel does14:56
NobodyCamahh14:56
lucasagomesdtantsur, not only that, we should consider root+swap+ephemeral14:56
NobodyCamdevananda: would you throw up ether pad with the comamnd you use to test translations14:56
lucasagomesthat's why I want the disk partitioning refactor to land so we can start tackling all these related problems with disk14:56
lucasagomesdtantsur, right now in trunk, it's very messy14:57
NobodyCamso I can learn14:57
NobodyCamhow to  too14:57
devanandahttps://etherpad.openstack.org/p/TestingTranslations14:57
dtantsurlucasagomes, is it possible to fix size calculation within your patch? For me it's plainly wrong to try to write partition at (N,N+1)14:58
*** dwalleck has joined #openstack-ironic14:59
lucasagomesdtantsur, yeah, so that patch is a short-term one14:59
lucasagomesfor the rc1 release only14:59
lucasagomescause the refactor is too big14:59
*** sphoorti_ has joined #openstack-ironic15:00
*** sphoorti has quit IRC15:00
dtantsurlucasagomes, that's ok, but I think we need to fix deployment for rc1 ;)15:00
dtantsurin the meanwhile, how can I work around?15:01
dtantsurmake flavor smaller?15:01
lucasagomesheh yeah :(15:01
devanandaNobodyCam: I think that's it15:01
devanandaHaomeng|2: https://etherpad.openstack.org/p/TestingTranslations15:01
NobodyCamwoo hoo.. and you get different output15:02
devanandayep15:02
devanandaNobodyCam: so that worked? was typing from last night's memory :)15:02
*** matty_dubs|gone is now known as matty_dubs15:02
NobodyCammind if I share in #openstack-translation ?15:02
agordeevdtantsur: it's partially work-arounded in devstack. https://github.com/openstack-dev/devstack/blob/master/tools/ironic/scripts/create-nodes#L14-L16 HTH15:02
devanandaNobodyCam: no -- please do15:02
NobodyCam:)15:03
devanandaI would like this to become automated15:03
devanandaso it doesn't break again15:03
* lucasagomes wants juno to opens15:03
lucasagomesopen*15:03
dtantsuragordeev, I see, thank you. So smaller flavor + smaller ironic node size.15:03
dtantsurthat's why things don't break for guys who use devstack15:03
devanandalucasagomes: me215:04
devanandalucasagomes: i'm confident now that we have translations at least for operator log files15:05
devanandaeven if the API isn't translated yet15:05
devanandahaomeng and i were looking into that ... but we're missing a middleware :(15:05
devanandaand my late-night attempts to hack one together failed15:05
lucasagomesdevananda, ah ouch!15:06
lucasagomesbut yeah it's very needed15:06
devanandayea15:06
lucasagomesand nobody seems to have noticed it was broken15:06
lucasagomesuntil u guys started looking into it15:06
lucasagomesso good work :)15:06
devanandacause testing of i18n is not done prior to packaging15:06
devanandawhich happens AFTER i tag the release15:06
devananda:(15:06
devanandaanyhow, i think the fix for API i18n will be small15:07
devanandaand contained in just a few files15:07
devanandaeasy backport if we/someone want it15:07
lucasagomesyay! tag me on the review when u get it up, I'll be happy to review that15:07
devanandalucasagomes: 74626 is mostly there15:07
devanandajust missing amiddleware15:08
lucasagomesand test (ironic in portuguese :D)15:08
devanandawoops15:08
lucasagomesdevananda, will take a look15:08
devanandawrong link15:08
devanandalucasagomes: https://review.openstack.org/#/c/84362/15:08
devanandathat's got all the pieces15:08
devanandajust the broken middleware15:08
devanandasmall enough I'm OK with backporting it15:09
devanandaso15:09
devanandaI'm goign to tag the release unless anyone stops me in the next 15 minutes :)15:09
NobodyCamI am still testing lucasagomes sfdidk fix15:09
NobodyCamsfdisk even15:10
lucasagomesdevananda, ack15:10
lucasagomesdevananda, btw, the fix that NobodyCam is testing is the quick fix for the swap >= 1G15:10
lucasagomesNobodyCam, :) thanks15:10
lucasagomes<=*15:11
lucasagomesurgh15:11
lucasagomes>=15:11
NobodyCamhehehe15:11
* lucasagomes confused15:11
openstackgerritTom Fifield proposed a change to openstack/ironic: Fix grammar in error string in pxe driver  https://review.openstack.org/8447115:11
*** sphoorti__ has joined #openstack-ironic15:11
* NobodyCam hands lucasagomes more coffee15:11
lucasagomesNobodyCam, cheers buddy hah15:11
lucasagomesI need it15:11
*** sphoorti_ has quit IRC15:12
NobodyCamwoo hoo patches from the i18n / L10n team15:12
openstackgerritTom Fifield proposed a change to openstack/ironic: Fix spelling error in conductor/manager  https://review.openstack.org/8447215:14
NobodyCamlucasagomes: undecloud deploying in process... sorry have to rebuild everything15:16
* NobodyCam makes coffee15:16
lucasagomes:D no worries15:17
openstackgerritA change was merged to openstack/ironic: Check that all po/pot files are valid  https://review.openstack.org/8421015:17
*** dwalleck has quit IRC15:20
devanandalucasagomes: link?15:22
NobodyCamlucasagomes: undercloud deployed and configured!15:23
mordredNobodyCam: w00t15:23
lucasagomesdevananda, https://review.openstack.org/#/c/84396/15:23
lucasagomesNobodyCam, yay!15:23
lucasagomesNobodyCam, thanks for testing :)15:23
NobodyCammorning mordred :)15:24
mordredmorning NobodyCam !15:24
BadCub01Morning mordred :-)15:24
BadCub01And Mornng Deva and NobodyCam15:25
*** ifarkas has quit IRC15:26
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Open Juno development  https://review.openstack.org/8447415:27
* lucasagomes wants to +2 ^ :P15:29
romcheglucasagomes: -2 is already there :)15:29
lucasagomesheh romcheg yeah15:29
NobodyCamlol15:29
lucasagomesdamn I was too slow15:29
lucasagomesheh15:29
devanandalol15:29
rloohow does python-ironicclient relate to the icehouse version of ironic?15:30
rloothe other clients seem to be released on a diff schedule.15:31
devanandarloo: yep, independent schedules15:31
devanandaI haven't seen any changes recently that requried a new release15:31
devanandabut imbw. let me know :)15:32
devanandalucasagomes: looking at https://review.openstack.org/#/c/84396/1/ironic/drivers/modules/deploy_utils.py15:32
devanandalucasagomes: I'd like more eyes on that15:32
devanandaas it's a pretty important change, and we have lots of history with sfdisk and none with parted15:32
rloolooks like python-ironicclient-0.1.2 was uploaded on 2014-02-25.15:33
lucasagomesdevananda, right, yeah I tested it locally here with the ephemeral, w/o ephemeral, 1mb swap, 2G swap, etc.. situations15:33
lucasagomesdevananda, yeah15:33
lucasagomesdevananda, NobodyCam also tested it in his last run of devtest15:33
lucasagomesI know it's risky, but currently sfdisk + 1MB lead partition is broken15:33
devanandalucasagomes: ack15:33
devanandabut it works with < 1GB swap15:33
devanandaso15:33
lucasagomesyeah15:34
devanandai'd also really like lifeless' input on it15:34
devanandaas he had a lot of opinions on the sfdisk patches originally15:35
NobodyCamI good with cutting the RC with current... that is was a release candidate is all about15:35
lucasagomesdevananda, +115:35
devanandaok15:36
devanandaso we have two items with backport potential15:36
devanandaAPI i18n and sfdisk->parted15:36
devanandait's good practice for me :)15:36
romcheg+115:36
NobodyCam:)15:36
rloowrt python-ironicclient, there are some requirements changes. (version updates for pbr, httplib2, keystoneclient, six)15:41
rlooand code changes related to console (which we don't care), --maintenance option for node-list, driver-show cmd15:43
NobodyCambbt... brb15:52
devanandalucasagomes: so having set-console-mode, get-console in the CLI when they're not in the API is a bit odd15:52
lucasagomesdevananda, they r in the api yes, no? /me checks15:53
devanandaah. they are.15:53
devanandanvm15:53
devanandajust no driver implements it15:53
lucasagomeshttps://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/node.py#L7315:54
lucasagomesyeah15:54
devanandayea15:54
devanandasorry :)15:54
lucasagomesnp15:54
*** mdurnosvistov1 has joined #openstack-ironic15:54
devanandarloo: lucasagomes: new 0.1.3 release of the client coming soon15:54
devanandajust pushed the tag to gerrit15:54
lucasagomesyay!15:55
lucasagomesthanks15:55
*** matty_dubs is now known as matty_dubs|lunch15:57
rloodevananda: thx (and lucasagomes too)15:58
*** romcheg has quit IRC16:00
rloodevananda: http://docs.openstack.org/developer/ironic/. It is still alpha quality right?16:02
devanandarloo: should probably be updated to say beta, heh16:03
devanandabut that's just my opinion16:03
rloodevananda: the only other thing is that it mentions baremetal in grizzly and havana, should probably say icehouse too. but seems minor.16:03
devanandayea16:03
rloodevananda: it is too late to change for icehouse rc, right?16:04
devanandano16:04
rloodevananda. well, we're going to be adding more docn anyway...16:04
devanandabut it's an online doc16:04
devanandaright16:04
*** eghobo has joined #openstack-ironic16:04
*** BadCub01 has quit IRC16:05
rlooyeah, on-line, and if someone actually wants to build the doc themselves, but let's assume not ;)16:05
NobodyCampost bbt walkies... bbiafm16:05
lucasagomessomeone knows any exception that can be returned in the spawn() method of nova16:12
lucasagomesto tell nova to _not_ try to re-scheduler the instance?!16:13
devanandanope16:13
lucasagomesI've tried DeployInstanceFailure, NovaException, InvalidParameterError16:13
lucasagomes:(16:13
devanandalucasagomes: but. look at nova/filters/retry?16:13
lucasagomesall then try to reshceduler it16:13
lucasagomes2014-04-01 17:09:21.635 TRACE nova.compute.utils [instance: 8ebc65de-45af-4b20-a074-6547246db027] RescheduledException: Build of instance 8ebc65de-45af-4b20-a074-6547246db027 was re-scheduled: test internal server error16:13
devanandait's the retry filter causing that16:13
lucasagomesah!16:13
devanandaso if there is a wa yto signal "dont retry" it'd be there16:13
lucasagomesdevananda, cheers will see16:13
*** jistr has quit IRC16:17
devanandayurg16:18
devanandacan't "nova delete" when the power is off16:18
devananda*when the provision state is none16:18
devanandaso if provision fails (as it does sometimes) then nova is stuck16:19
devanandaironic won't let it delete teh instance16:19
lucasagomeswell seems there's none16:19
lucasagomesdevananda, heh that's what I'm fixing16:19
devanandagreat16:19
lucasagomesdevananda, https://bugs.launchpad.net/ironic/+bug/130028816:19
devanandai was trying to test your parted patch16:20
lucasagomesdevananda, right, any errors on the ir-cond related to the parted stuff?16:20
NobodyCamand back :)16:20
jrolllucasagomes: if you can't figure it out, probably worth asking comstud about it16:22
jrollwrt nova stuff16:22
lucasagomesjroll, yeah16:23
lucasagomesjroll, I took a look at the filter it seems to retry anyway, doesn't even check for a particular exception16:23
lucasagomesbut I will ask in #openstack-nova16:23
jrollyeah, that works too16:23
devanandalucasagomes: none yet, since it failed before then16:24
lucasagomesjroll, thanks, asked there and pinged comstud :)16:24
jrollwhy do we not want nova to reschedule it? because ironic will retry?16:24
jrollor because we should assume the node is broken somehow?16:24
lucasagomesjroll, if the deploy() fails in things like validation of the driver parameters16:24
lucasagomesthere's no reason to retry it16:25
*** mdurnosvistov1 has quit IRC16:25
jrollyeah16:25
devanandawell16:25
devanandamaybe that node's driver_info is bad16:25
devanandaand another node will work16:25
devanandathere are cases where retry is good16:25
*** dwalleck has joined #openstack-ironic16:25
devanandawe can allow the driver to make some decisions there16:25
jrollI'm wondering if we should put the provision state in something other than None16:25
*** dwalleck_ has joined #openstack-ironic16:25
lucasagomesdevananda, hmm yeah, but driver_info is populated by the nova ironic driver no?, depends on the driver16:25
lucasagomesyeah there's many corner cases16:26
*** mdurnosvistov1 has joined #openstack-ironic16:26
devanandalucasagomes: so, not entirely, and not for long16:26
devanandalucasagomes: we're going to move all the stuff which nova is writing to driver_info today, into instance_info16:26
lucasagomesyeah16:26
devanandajroll has a patch up for that, iirc16:26
* lucasagomes wants the deploy_info stuff!16:26
jrolldevananda: I have a patch up to create the instance_info field16:27
jrollnothing up to move the data yet16:27
devanandaright. that'll be in the nova driver16:27
jrollright16:27
*** dwalleck has quit IRC16:29
*** dwalleck has joined #openstack-ironic16:31
*** dwalleck_ has quit IRC16:33
devanandalucasagomes: oh, also, looks like nova delete is keying on the stored node_uuid16:35
devanandalucasagomes: rather than the instance_uuid16:35
*** dwalleck_ has joined #openstack-ironic16:35
devanandalucasagomes: when I manually unassociated it in ironic, nova's still calling do_node_teardown on the node!16:35
lucasagomesdevananda, yeah, I have to change the tear_down to check which state the provision_state of the node is16:36
lucasagomesso based on that it then issue the request to tear down the node16:36
devanandalucasagomes: and to qery by instance_uuid16:36
lucasagomesdevananda, yup16:36
lucasagomesnode = validate_instance_and_node(icli, instance)16:37
lucasagomesit tries to get the node using the instance_uuid16:37
*** dwalleck has quit IRC16:37
lucasagomesif it's not there we just return16:37
lucasagomescause it's a non-existent instance16:37
lucasagomesI will try to push my patch soon16:37
openstackgerritPablo Fernando Cargnelutti proposed a change to openstack/ironic: Overwriting node_is_available in IronicDriver  https://review.openstack.org/8421416:37
devanandaright now any attempt i'm making to boot16:39
devanandayields this16:39
devanandahttp://paste.openstack.org/show/74762/   <-- NobodyCam, lucasagomes16:39
* NobodyCam looks16:39
* lucasagomes checks16:39
NobodyCamdevananda: did you move the driver location?16:40
NobodyCam/opt/stack/nova/nova/virt/ironic/driver.py16:40
NobodyCamshould that not be /opt/stack/ironic/nova/virt/ironic/driver.py16:41
*** ndipanov has quit IRC16:41
*** athomas has quit IRC16:41
lucasagomeslooks like an old error16:41
lucasagomesdevananda, https://review.openstack.org/#/c/78686/16:42
devanandahrmm16:43
*** derekh has quit IRC16:44
devanandaah. old .pyc files in nova.virt.ironic16:48
NobodyCamdoh16:49
* lucasagomes hate when it happens 16:49
*** hemna_ has joined #openstack-ironic16:51
devanandaHTTPBadRequest: Couldn't apply patch '[{'path': '/extra/vif_port_id', 'op': 'remove'}]'. Reason: u'vif_port_id'16:52
devanandaok, other than that, deploy seems to be working16:52
NobodyCamdevananda: vif_port_id not set on that node?16:52
devanandaNobodyCam: ive seen that error in all the recent virtual-ironic jenkins test runs16:54
*** BadCub01 has joined #openstack-ironic16:57
*** matty_dubs|lunch is now known as matty_dubs16:57
*** stevehuang has joined #openstack-ironic16:59
*** dwalleck_ has quit IRC17:01
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Reduce logging output from non-Ironic libraries  https://review.openstack.org/8449617:04
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Fix Nova rescheduling tear down problem  https://review.openstack.org/8449717:04
Shrewssoooo much easier to grok the logs with that ^^^^, but open to adding some back if others find the noise useful17:04
lucasagomesdtantsur, ^17:05
devanandaShrews: some of those might be better left at INFO17:06
devanandaShrews: but I agree that DEBUG is too noisy17:06
Shrewsdevananda: i didn't see any cases where they output anything at INFO level17:06
Shrewsdevananda: but it's easy to miss it as it is17:06
lucasagomesShrews, devananda +1 very noise yeah17:06
devanandalucasagomes: ok. so you have 2 patches up that are both pretty important fixes17:07
lucasagomesdevananda, right17:07
devanandawell. honestly. we have a lot of fixes ...17:07
lucasagomes+117:07
NobodyCamreview jam tomorrow morning?17:07
lucasagomesthe i18n stuff also very important for the release17:08
lucasagomesNobodyCam, we can do that, I mean, I would like to have a test jam!17:08
lucasagomesheh17:08
devanandalucasagomes: API or LOG i18n? or both17:08
lucasagomesI guess both17:08
devanandacause we have LOG i18n now17:08
lucasagomesthat patch u pointed me to? /me have to test17:08
devanandathat patch is for APi i18n17:09
lucasagomesah right, I haven't tested i18n at all, so... :(17:09
lucasagomesok I will test the api i18n17:09
devanandait's gonna break. the middleware is busted in that WIP17:09
lucasagomes:/17:10
devanandaso when i said i was gonna tag it unless anyone objects, you uploaded more patches and I see you targeted teh parted to RC117:11
devananda(maybe you did that before)17:11
devananda:)17:11
devanandalucasagomes: should I take that as an answer, you want me to hold the tag?17:11
devananda*hold off on17:11
lucasagomesheh I did that before17:11
devanandaah17:12
lucasagomesbut, hmmmm idk it sounds like an important fix17:12
devanandayea17:12
devanandai agree17:12
lucasagomespartitioning broke really sucks17:12
devanandaafter fixing up my env issues, parted worked for me with 2GB swap17:12
lucasagomesbut I understand the risk as well17:12
lucasagomes:)17:12
lucasagomesyeah17:12
devanandaexample of my concer - has anyone tested parted on 12.0417:12
lucasagomesso I tested here locally many combinations, but you never know17:12
lucasagomesI haven't :(17:13
devanandasince that's waht's in the gate17:13
lucasagomesmaybe the devstack guys17:13
NobodyCambrb17:13
lucasagomesI've a ubuntu vm here but I think it's not 12.04, lemme check17:14
*** harlowja_away is now known as harlowja17:14
lucasagomes13.04 :/17:14
devanandai hesitate to tag when we already have a fix up. why do a backport if we can just wait a few more hours to land this?17:14
lucasagomesyeah17:15
lucasagomesmakes sense to hold a bit more17:15
devanandalifeless: when you're up, I would like your input on https://review.openstack.org/#/c/84396/1 -- this (new patch) is holding up the RC, and I think you had expressed opinios on parted/sfdisk/etc things in the past.17:16
lucasagomesdevananda, better wait until tomorrow tops? then we can see lifeless opnions about it17:17
lucasagomesotherwise we can tag it and if it's _really_ needed we backport17:17
lucasagomesis good that there's a plan b17:17
devanandayea17:17
devanandaok. things work, just with limitation right now17:18
lucasagomesyeah17:18
devanandalucasagomes: if you agree, would you mind commenting / untargeting the bug?17:18
lucasagomesdevananda, sure, no objections17:18
devanandagreat17:18
JayFFWIW parted as old as lucid iirc supports MiB17:19
JayFI've used it since 10.04 with that flag17:20
lucasagomesdevananda, done17:20
lucasagomesJayF, nice it's a good feedback, thanks17:22
JayFParted is great. And it's good to move to it (even if in Juno) because it supports GPT whereas sfdisk doesn't, and GPT is required for extra-large boot devices among other things17:22
lucasagomesexactly17:22
lucasagomes>2TB partitions will need gpt17:23
NobodyCamyes17:23
lucasagomesthe idea of the refactoring was to enable that as well17:23
JayFI used parted here http://git.openstack.org/cgit/openstack/ironic-python-agent/tree/ironic_python_agent/shell/copy_configdrive_to_disk.sh#n25 because it lets you set a negative offset to put something at the end vs the beginning of a disk, was super useful for that17:23
lucasagomesnice!17:24
lucasagomesJayF, for the agent you guys might want to use the python-parted libs as well17:25
lucasagomesI've worked with it in another project (https://github.com/umago/carbono)17:25
lucasagomesit's quite useful17:25
JayFI'm sure we'll work all that shell stuff out of the agent over time17:25
JayFwas just a quick and easy way to interact with the disks for getting something working fast17:25
lucasagomesyeah I understand17:26
devanandalucasagomes: approved. we should see it merge soon17:30
devanandalucasagomes: at which point ttx will branch, and we should retarget your fix to that branch17:31
devanandalucasagomes: and land all the diskpartitioning things on master17:31
devanandaand al lthe other things17:31
openstackgerritA change was merged to openstack/ironic-python-agent: Add kernel parameter support  https://review.openstack.org/8426417:31
*** dwalleck has joined #openstack-ironic17:34
NobodyCambrb17:38
lucasagomesdevananda, yay! ack17:42
lucasagomeswill retarget17:42
lucasagomesdevananda, all the other things you mean, get rid of swap, etc?17:42
lucasagomeswhole series?17:42
lucasagomesserie*17:43
devanandayes17:43
lucasagomesdevananda, ah I see, approved the tag u mean?17:43
lucasagomesnice ok17:43
devanandarc1 status page was green :)17:44
NobodyCamI have tested them.. (not with > 1 gb of awap thou17:44
NobodyCam)17:44
NobodyCamwas17:44
devanandafor the i18n stuff, i added some more notes to https://etherpad.openstack.org/p/TestingTranslations just now17:44
devanandafor testing the API17:44
lucasagomesso if we are going to approve the series that short-term can be abandoned17:44
devanandalucasagomes: no...17:44
devanandalucasagomes: well. it can be. yes.17:45
NobodyCamthere are questions out to lifeless on the parted patches no?17:45
lucasagomesyeah, that waas a quick fix for the rc1 release only17:45
devanandabut I would prefer you leave it up as a WIP or target it to rc217:45
lucasagomesNobodyCam, yeah17:45
devanandawe can have more RCs, and folks may need things back ported17:45
lucasagomesdevananda, ack will WIP it out17:45
NobodyCamlol :)17:46
devanandai'm not sure exactly how it will go. first time we've done a release with folks using it :)17:46
* NobodyCam heard devo in his head17:46
lucasagomesack :)17:46
lucasagomeswipped17:46
devanandathanks!17:46
ShrewsNobodyCam: ugh17:46
devanandalifeless: pls disregard my earlier ping. unless you just want to talk about parted vs sfdisk in general :)17:47
ShrewsNobodyCam: that one song makes me wish i had chosen a different term than "work in progress" when i implemented that17:47
NobodyCamhttps://www.youtube.com/watch?v=j_QLzthSkfM17:47
NobodyCamlol17:48
lucasagomeslifeless yeah if there's objects about parted please put at https://review.openstack.org/#/c/8339617:48
lucasagomesobjections*17:48
lucasagomesack I'm done for today then17:48
* lucasagomes tired17:48
NobodyCamhave a good night lucasagomes :)17:48
lucasagomeshave a g'night devananda NobodyCam everyone :)17:48
NobodyCamenjoy a pint for me17:49
*** lucasagomes is now known as lucas-afk17:49
lucas-afkNobodyCam, heh will do :)17:49
devanandag'ngiht lucas-afk !17:50
Shrewsadam_g: fyi, giving your nova driver tests change a test drive17:56
adam_gShrews, cool :)17:58
*** sphoorti__ has left #openstack-ironic17:58
openstackgerritA change was merged to openstack/ironic: Open Juno development  https://review.openstack.org/8447417:59
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Add BackOffLoopingCall with jitter  https://review.openstack.org/8430318:00
*** dwalleck_ has joined #openstack-ironic18:02
*** dwalleck_ has quit IRC18:03
JoshNangwoo Juno!18:04
NobodyCamw00t!!!118:04
*** dwalleck_ has joined #openstack-ironic18:04
*** romcheg has joined #openstack-ironic18:04
devanandalet the flood gates open, heh18:05
*** dwalleck has quit IRC18:06
ShrewsI'm bored with Juno already. Can't we just move on to the Kardashian release already?18:07
*** derekh has joined #openstack-ironic18:07
devananda:p18:08
devanandaShrews: just kidding. really i want you to work on backporting all of ironic to essex.18:08
devanandaShrews: can you do that by friday? kthx :)18:09
devananda</kidding>18:09
Shrewsoh, sure. no problem.18:09
Shrewsin related news, mordred, i quit18:09
Shrews:)18:09
* NobodyCam looks at the 4/1 date on his cal and just shakes his head :-p18:10
jrollis there any known reason that one out of every 2-3 requests would fail with "No valid host was found. Reason: No conductor service registered which supports driver teeth"?18:11
jroll(given that I have a conductor running with the teeth driver loaded)18:11
jrollmaybe a conductor did not unregister properly?18:12
devanandajroll: neat. um18:12
* mordred throws an angry wet cat at Shrews head18:12
devanandajroll: look at the log for said conductor18:12
devanandajroll: also run "ironic driver-list" to see what drivers are being advertised18:13
jrolldevananda: I don't see anything in the conductor logs around the time of failure18:13
devanandajroll: it could be before that18:13
devanandajroll: that error comes from API side when it tries to map a requested node to the hash ring, and can't find a conductor that handles that driver18:14
devanandajroll: eg, the conductorw ith 'teeth' driver isn't updating its heartbeat18:14
jrolldevananda: right18:15
jrolldevananda: I have to run off for now, but that points me in the right direction. thanks18:15
* NobodyCam makes a bagel18:16
*** dwalleck_ has quit IRC18:24
NobodyCamdevananda: +2 the parted patches.18:32
devanandammmm, macaroni ....18:45
NobodyCam:)18:46
ShrewsDid I see someone post something about a 'flavor_get' attribute error earlier? Could have swore I did, but not finding it in any srollback18:49
devanandaShrews: yea, i posted it.18:49
devanandaShrews: problem was old .pyc files in nova/nova/virt/ironic/18:49
Shrewsdevananda: i shouldn't be seeing that in a freshly cloned nova branch18:51
Shrewswhich is what adam_g's change is doing to run the unit tests18:52
adam_gShrews, hows that working for you?18:53
Shrewsadam_g: it's not18:53
ShrewsAttributeError: <nova.virt.fake.FakeVirtAPI object at 0x7be64d0> does not have the attribute 'flavor_get'18:53
adam_gShrews, did you pull both patches?18:53
Shrewsadam_g: oh, there are 2?18:54
* Shrews looks18:54
adam_ghttps://review.openstack.org/#/c/84033/18:54
adam_ghttps://review.openstack.org/#/c/84043/18:54
adam_gthe unittests needed updating to account for that flavor_get change18:54
adam_gas well as other things we've changed in driver18:54
Shrewsadam_g: ok thx. retrying18:54
Shrewsadam_g: perhaps those changes should be combined18:55
NobodyCamdevananda: added TestingTranslations to the whiteBoard18:55
NobodyCam(etherpads)18:55
adam_gShrews, they could. i was expecting to make some tweaks to the nova-tests.sh script separate from the tests tho18:56
adam_gmay not be needed tho18:56
Shrewsadam_g: ok, so that works18:59
adam_gShrews, cool18:59
devanandai'm gonna be semi-afk for the rest of the day19:00
devanandaaround for the release and meetings19:00
devanandabut have some offline meetings as well19:00
Shrewsadam_g: any reason you don't invoke the tests through tox?19:00
*** jistr has joined #openstack-ironic19:01
NobodyCamhave a good rest of the day devananda :)19:01
adam_gShrews, AIUI tox commands will execute from the top level of ironic tree. we need to execute nova's setup.py from within its tree so testr only loads nova code + tests.19:02
adam_gShrews, also i'm expecting we will need to adjust as that script a bit for it to fit into the unit test gate jobs, in terms of generating and storing  its storing its subunit results, etc.in a place that the jenkins job expect19:03
NobodyCamadam_g: would tox -e py27 -- ironic.nova.tests work?19:03
Shrewsadam_g: or 'cd $NOVA_REPO && tox -epy27'19:04
Shrewstox reads the tox.ini from current dir, iirc19:04
adam_gNobodyCam, i dont think it would. thats what i originally wanted--but testr will go and load *all* of the tests in ironic, which cause an import of both ironic.* and nova.*, and there are issues there WRT oslo config resources being stomped on19:04
NobodyCamahh19:05
adam_gShrews, oh--your original question was about why nova-tests.sh invokes setup.py direclty instead of tox?19:05
Shrewsadam_g: correct19:06
adam_gShrews, yeah-i think that would probably work too19:06
Shrewsit would abstract the test runner away19:07
adam_gShrews, are we still able to pass positional arguments down to setup.py/testr?19:07
Shrewsadam_g: yes19:07
adam_gim not sure we want to run the entire nova suite or just the ironic tests19:07
Shrewsadam_g: tox -epy27 -- my_arguments19:07
* adam_g gives it a shot19:08
*** blamar has joined #openstack-ironic19:08
Shrewsadam_g: examples in the quick start: http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html#19:08
*** romcheg has quit IRC19:08
Shrewsadam_g: you can select individual tests, or pass testr args19:09
adam_gShrews, right19:09
*** jbjohnso_ has quit IRC19:10
*** eguz has joined #openstack-ironic19:11
adam_gShrews, so that ends up creating a nested venv19:11
Shrewsadam_g: it creates a venv that tox will use to run the tests. that should be ok19:11
Shrewsi'd rather see that than have my personal dev env get mucked around with19:12
adam_gShrews, how so? everything is contained in the .tox/nova-tests/ venv19:12
adam_gas is now, at least19:12
Shrewsadam_g: i think you could eliminate the whole in_env stuff19:12
Shrewsadam_g: sorry, yeah, it's not doing it now (mucking around with your in_venv environment).19:14
Shrewsadam_g: hrm, i see your point about the nesting though19:15
*** eghobo has quit IRC19:15
adam_gShrews, also, relying on nova's tox means the entire execution happens outside of ironic's tox--you'd run tools/nova-tests.sh only.19:15
NobodyCambrb...19:15
Shrewsadam_g: i was thinking that nova-tests.sh would call tox instead of testr, but now that i'm thinking about it, i'm seeing more disadvantage than advantage to doing that19:16
adam_gShrews, this'll require a bit of tweaking here and in infra to have this running as part of the gate pipeline. making it executable from ironic as 'tox -e nova-tests' will probably cause the least ammount of friction getting it in there19:17
Shrewsyeah, never intended to change that part19:17
Shrewsadam_g: leave it as-is for now, is my suggestion19:17
* Shrews needs to stimulate his brain with chai tea19:18
JayFdevananda: +1 GF Macaroni and Cheese :P19:19
JayFdevananda: if you're ever back in the Bay Area (idk where you live), make it down to Jack's in San Bruno for GF Bacon Mac. Mmmm19:19
NobodyCamJayF: ummmmmm.... yummy19:26
JayFthey have regular Bacon Mac. So don't worry. You can have wheat :P19:27
NobodyCamheheheh :)19:27
comstudargh19:29
NobodyCam:( ?19:29
comstudCan someone answer a question for me regarding TaskManager() __init__() ?19:29
comstudI guess I can search the code, but..19:29
comstud137         :param node_ids: A list of ids or uuids of nodes to lock.19:29
devanandaJayF: seattle. I get t othe bay fairly often tho19:29
comstudWhy can that be either?19:29
comstudin here is:19:29
comstud167                     node = self.dbapi.get_node(id)19:29
comstudwhich will only search by uuid19:30
devanandacomstud: there's some magic in the dbapi19:30
devanandacomstud: it actually takes either one19:30
devanandacomstud: optimization for databases. if you already know the 'id', it's faster to look up that way19:30
NobodyCamcomstud: id are never expoded to users19:30
NobodyCambut are used internally19:30
comstudyeah I see19:30
comstudbut that's very inconsistent19:30
devanandacomstud: internally or w/ othe rprojects?19:31
comstudand the object version of Node only has a 'get_by_uuid' method19:31
comstudwhich I guess should just be 'get'19:31
comstudsince it can take either.19:31
*** jbjohnso_ has joined #openstack-ironic19:31
devanandahmm. yes.19:31
NobodyCam:-p19:31
comstudI'm just speaking in general19:31
devanandaname is inconsistent there19:31
comstudI'm fixing up a bunch of object stuff and ran into this19:31
*** max_lobur1 has quit IRC19:31
NobodyCam++19:31
*** max_lobur has joined #openstack-ironic19:31
comstudand I generally hate "well this variable can be a list of x or a list of y"19:31
comstudbut I can deal with it, I guess19:32
NobodyCam:)19:32
devanandacomstud: you'll hate that TaskManagers can lock "a node" or "a list of nodes" then19:32
NobodyCam:-p shhhhh...19:33
comstudI can deal better with a singleton vs a list19:33
devanandaheh19:33
devanandak19:33
comstudassuming they are the same type19:33
comstudi mean, list member is same type as singleton19:33
comstudheh19:33
comstudbut anyway :)19:33
comstudOk.19:33
comstudalso interesting is..19:34
comstudNode.get_by_uuid calls dbapi.get_node, which returns an object19:34
comstudand Node.get_by_uuid re-turns it into an object itself19:34
comstudSo, here's what I want to do.. but I don't know if you agree..19:34
comstudI'd like objects.Node.get() be the interface you use..19:35
comstudinternally it'll use the dbapi, but the dbapi itself doesn't need to form the object then19:35
Shrewsadam_g: so, i left my observations on that review. In case I wasn't clear, only point #3 is a stickler for me.19:36
comstudThe related problem is your objectify() decorator actually should be passing a context to _from_db_object()19:36
Shrewss/wasn't clear/wasn't clear in my comments/19:36
comstudyet noone of the dbapi calls have context on them19:36
comstudnone19:36
comstudso to clean this up, I either: add context to all of the dbapi methods, or remove the objectify decorator19:37
comstudI'd rather do the latter.. and if the objects code is the interface you use, it is what makes sure to return the object version itself19:37
NobodyCamcomstud: what I'm tring to think about is would having context in the dbapi be useful downthe road19:39
comstudI find it kind of pointless and at the wrong layer19:39
comstudthe object calls will all have context and can do things with it there19:40
comstudwhether it be telling the db layer to apply certain filters or what19:40
NobodyCamvery true19:40
comstudWe have them in nova on the dbapi calls, but a lot of it is kinda dumb... we require admin context for certain dbapi calls, yet we have some policy checks for them also19:41
comstud(up higher)19:41
devanandacomstud: latter19:42
NobodyCamcomstud: while at this point I am not 100% there I am also NOT opposed to your suggestion19:42
openstackgerritA change was merged to openstack/ironic: Fix spelling error in conductor/manager  https://review.openstack.org/8447219:42
comstudwell, I'll be throwing up some patches... it'll be easier to see there19:42
devanandacomstud: afaict we shouldnt ever need context on the dbapi calls19:43
comstudand we can debate in the reviews19:43
NobodyCamoh devananda while I have you.19:43
devanandacomstud: noav uses that for permanent delete, view deleted stuff, etc.19:43
comstuddevananda: that's what I would hope19:43
devanandacomstud: and ironic doesn't keep history like that19:43
devanandacomstud: access should be checked at the API tier only, IMO19:43
comstudnova also filters by project for getting lists of instances19:43
comstudand so forth19:43
comstudbut19:43
devanandaah19:43
NobodyCamis htere a list of patches that I can ping the owners of to re-submitt now that J is offically open?19:43
comstudsome of that now is actually passed in to dbapi as 'filter this project'19:43
comstudwhich I prefer19:43
comstudat a higher layer19:44
comstudkeep the db layer a little more stupid19:44
devanandawe could eventually have projects in ironic. but yes ^19:44
comstudcools19:44
devanandaNobodyCam: no. but go down this page https://review.openstack.org/#/q/status:abandoned+project:openstack/ironic,n,z19:44
NobodyCamdevananda: quick after noon walkies then I'll ping the -2'd patch owners :)19:46
*** florentflament has quit IRC19:51
comstuddevananda: hm, yeah, I think interestingly...19:56
comstudyou could currently pass an id like '1' to ironic API...19:56
comstudand it'll happily fetch that node19:56
NobodyCamcomstud: it should...19:56
NobodyCamit used to19:56
NobodyCamlol19:56
NobodyCamjust dont tell anyone19:57
comstudYou think it *should*?19:57
comstud:)19:57
comstudHehe19:57
devanandano19:57
NobodyCamI have not tested that in quite a while19:57
devanandait should not19:57
comstudok19:57
devanandai believe we fixed that19:57
NobodyCamoh19:57
devanandaGET /v1/nodes/119:57
comstudI don't see how it can be fixed19:57
NobodyCamwe may have19:57
devanandathis ^ should fail19:57
comstudsince you call Node.get_by_uuid() which calsl dbapi.get_node()19:57
devanandacomstud: enforcing in teh API layer that a UUID is received19:57
devanandatype cehcking in wsme19:57
NobodyCamahh that is right...19:58
comstudyeah19:59
comstudok.19:59
devanandahttps://launchpad.net/ironic/icehouse/icehouse-rc120:00
devanandaReleased: 1 minute ago20:00
devanandaheh20:00
*** romcheg has joined #openstack-ironic20:00
JayF\o/20:01
NobodyCamw00t \o/20:01
NobodyCam┏(-_-)┛┗(-_- )┓┗(-_-)┛┏(-_-)┓20:02
*** derekh has quit IRC20:05
*** jbjohnso_ has quit IRC20:07
*** mdurnosvistov1 has left #openstack-ironic20:07
uberjdevananda: does that mean ironic will be in icehouse?20:11
dividehexI want to know also ^^20:12
JayFAIUI, Ironic is still incubating, it's not graduating in Icehouse. We20:13
JayF*We're just following the release process in good faith and so if someone chooses to use Ironic with Icehouse, it's stablized for that purpose.20:13
uberjthanks, JayF20:14
devanandaJayF: that's correct20:18
*** romcheg has quit IRC20:19
*** florentflament has joined #openstack-ironic20:24
*** eguz has quit IRC20:27
*** mrda_away is now known as mrda20:27
mrdamorning all20:27
*** eghobo has joined #openstack-ironic20:28
NobodyCammorning mrda20:29
mrdaHey NobodyCam20:29
adam_gdevananda, https://review.openstack.org/#/c/84543/20:33
*** romcheg has joined #openstack-ironic20:35
*** jistr has quit IRC20:37
openstackgerritAdam Gandelman proposed a change to openstack/ironic: Add tox target + script for nova driver tests  https://review.openstack.org/8403320:39
NobodyCamdevananda: there is a list on the bottom of : https://etherpad.openstack.org/p/IronicWhiteBoard (Patch requested for re-submission now that Juno is Open) could you remove your -2's when you have a free minute20:40
openstackgerritRuby Loo proposed a change to openstack/python-ironicclient: Add main developer page  https://review.openstack.org/8454520:44
openstackgerritMichael Davies proposed a change to openstack/ironic: Handling validation in conductor consistently  https://review.openstack.org/8437420:46
NobodyCamlol fyi: Atlanta has very few rv parks :-p20:48
devanandaNobodyCam: ah. later today or tonight. no time right now20:49
NobodyCamnp list is there for you :)20:50
NobodyCamthat sould be all of them20:50
*** florentflament has quit IRC20:55
*** jdob has quit IRC20:57
*** blamar has quit IRC20:58
*** blamar has joined #openstack-ironic21:00
openstackgerritRuby Loo proposed a change to openstack/python-ironicclient: Add main developer page  https://review.openstack.org/8454521:04
*** blamar has quit IRC21:04
*** blamar has joined #openstack-ironic21:06
*** linggao has quit IRC21:11
*** blamar has quit IRC21:13
*** blamar has joined #openstack-ironic21:14
comstudman21:14
comstuda number of modules are doing node.get()21:14
comstudwhen it should be the fricken node object21:14
comstudthe tests are testing passing dicts21:16
comstudand not the objects.21:16
*** blamar has quit IRC21:18
*** blamar has joined #openstack-ironic21:18
mrdacomstud: sounds like an opportunity :)21:20
*** blamar_ has joined #openstack-ironic21:20
comstudyeah, well, it has to be fixed as a part of what i'm doing21:21
NobodyCamcomstud: how large is the patch getting?21:21
comstudwell, there's going to be quite a number of patches21:21
comstudI'm only fixing get_node* and related things right now21:21
NobodyCam:)21:21
comstudit's not that bad21:22
comstudso far, +218, -13821:22
comstudbut i still ahve 90 test failures21:22
comstud:)21:22
NobodyCamoh not bad at all,,21:22
NobodyCam:)21:22
comstudi think there's something bad going on with the ports API tests also21:23
*** blamar has quit IRC21:23
*** blamar has joined #openstack-ironic21:25
*** blamar_ has quit IRC21:25
NobodyCam:-/21:25
devanandahmm21:25
comstudI only happened to uncover these issues because I created a 'get' method on Node21:25
comstudheh21:25
devanandaso we'll need an oslo liason -- https://wiki.openstack.org/wiki/Oslo/ProjectLiaisons21:25
comstudwhich overrides the dict.get21:25
comstudsyntax21:26
comstudwhich no one should do with objects21:26
comstudjust access the field, mann21:26
comstud  File "ironic/drivers/modules/pxe.py", line 94, in _parse_driver_info21:26
comstud    info = node.get('driver_info', {})21:26
comstudas an example21:26
comstudand of course, when I fix that...21:27
comstudI find the tests are passing a dict.21:27
comstudand not the Node object21:27
comstud:)21:27
*** dwalleck has joined #openstack-ironic21:27
comstudactually, turns out the pxe module tests are correct.. but the seamicro and ssh ones were not21:30
comstudwell anyway, fun21:31
NobodyCamrloo: good catch with bug 1300021:32
NobodyCam1300990 even21:32
NobodyCam:-p21:32
rlooNobodyCam: Once that gets in, wait til you see the docn I'm going to write (or not) ;)21:34
comstudis ironic/common/context used?21:38
comstudthe tests, of course, mostly use ironic/opestack/common/context21:38
NobodyCamrloo: looking at conf.py on 8454521:41
NobodyCamusing ironic as reff. i see this: https://github.com/openstack/ironic/blob/master/doc/source/conf.py#L1721:42
NobodyCamsee any reason to add that?21:42
rlooNobodyCam: the wsme_protocols? Do you know what that is used for?21:42
rlooNobodyCam: I don't think ironicclient uses wsme anything.21:43
rlooNobodyCam: so I don't think it is needed. I figure we'll find out if there are errors, but I didn't see any related to wsme ;)21:44
NobodyCamI saw it on this review: https://github.com/openstack/ironic/commit/30206795b71d9b53c586385fe235d588bfe70a4421:44
NobodyCamjust need to test the patch here :)21:45
rlooNobodyCam: the client doesn't use pecan or wsme. I was wondering what httpdomain was for but it looks like used with pecanwsme.21:47
NobodyCamdevananda: do you happen to recall adding this https://github.com/openstack/ironic/blob/master/doc/source/conf.py#L1721:47
devanandanope21:48
NobodyCam:-p21:48
rlooNobodyCam: wrt sphinxcontrib.httpdomain, ironicclient doesn't need it cuz no RESTful api there: http://pythonhosted.org/sphinxcontrib-httpdomain/21:49
*** matty_dubs is now known as matty_dubs|gone21:51
*** dguerri_ has joined #openstack-ironic21:53
*** mtaylor has joined #openstack-ironic21:53
*** dguerri has quit IRC21:54
*** openstackgerrit has quit IRC21:54
*** mordred has quit IRC21:54
*** romcheg has quit IRC21:56
NobodyCamrloo: lgtm22:00
NobodyCambrb22:02
comstudOk22:03
comstudI found a case where the API will take a node ID22:03
comstudthe API Port declaration has:22:04
comstud80     node_uuid = wsme.wsproperty(types.uuid, _get_node_uuid, _set_node_uuid, 81                                 mandatory=True)22:04
comstudand _get_node_uuid doesn't care if it's an ID or a UUID22:04
comstudand there seems to be some tests passing an integer somehow22:04
rlooNobodyCam: thx!22:06
* NobodyCam thinks comstud is digging up all our skeletons 22:06
NobodyCamhehehe22:06
NobodyCam:-p22:06
comstudyeah22:06
comstudhm.22:07
comstudPOST: /v1/ports {'node_uuid': '1be26c0b-03f2-4d2e-ae87-c02d7f33c123', 'uuid': '1be26c0b-03f2-4d2e-ae87-c02d7f33c781', 'address': '52:54:00:cf:2d:31', 'extra': {'foo': 123}}22:07
comstudthat is somehow causing a node lookup of 12322:07
rloocomstud: thx for finding that; i think it is a bug ;)22:07
NobodyCamthat iindeed sounds like a bug22:08
comstudhttp://paste.openstack.org/show/74776/22:09
comstudthat last line22:09
* NobodyCam shutters but clicks any way22:10
comstudi'm not sure where 'node_id' is coming from22:10
comstudis extra supposed to be a dict of key to node ids?22:11
NobodyCamno22:11
mrdacomstud: better to find those bugs now :)22:13
comstudah i see22:14
comstudcreate db record, node_id is set22:14
comstudand that's used to create Port()22:15
comstudi wonder if: setattr(self, 'node_uuid', kwargs.get('node_id'))22:15
comstudis supposed to be node_uuid22:15
comstudnot node_id22:15
comstudalthough there's no node_uuid i guess in the DB22:15
comstudor is there22:15
comstudhm22:15
comstudno22:16
comstudnot sure how to fix this, going to leave it supporting both in my patch22:16
comstudand file a bug22:16
devanandacomstud: i love that you're digging up all our skeletons :)22:17
NobodyCam:)22:17
devanandagotta run... bbl22:17
comstudhehe22:17
comstudI'm not sure I'm loving it22:18
comstudbut at least someone is22:18
comstud:)22:18
NobodyCamcomstud: but we appreciate so much :)22:18
*** jgrimm has quit IRC22:23
comstudhttps://bugs.launchpad.net/ironic/+bug/130104622:25
*** radsy has joined #openstack-ironic22:29
NobodyCamWoo Hoo get to spend the night in Roswell New Mexico on the way to Atlanta... :-P22:32
*** lucas-afk has quit IRC22:41
NobodyCamanteaya: wow that was quick...22:41
comstudwe'll see how this looks in gerrit..22:43
comstudbot die?22:43
comstudhttps://review.openstack.org/#/c/84573/22:44
NobodyCammay just be slow22:44
NobodyCamopenstack user still in channel22:44
anteayahello22:45
NobodyCamhi anteaya :)22:45
anteayaI dislike spending the night in roswell new mexico22:45
anteayaI spent one night there and I wouldn't do so again voluntarily22:45
NobodyCamanteaya: you've done it before?22:46
NobodyCamlol :)22:46
anteayadreampt that my father was being coorerced by aliens and was trying to coorce me22:46
NobodyCammay I or SHOULD I ask why22:46
anteayaso I got up and cleaned the energy of the room for the next few hours22:46
anteayathat is why22:46
anteayanever had that kind of dream before or since22:46
NobodyCamahh22:47
NobodyCam:)22:47
anteayait was when I drove with them down to tucson22:47
anteayaso what was I quick about?22:47
anteayaI like new mexico22:48
anteayathe white sands are nice22:48
NobodyCamthe confirmed replay to the Ironic PTL email22:48
anteayaah22:48
anteayaI was watching22:48
NobodyCamhehehh :)22:48
NobodyCamwhen is voting?22:48
anteayastarts friday22:48
NobodyCam:)22:48
anteayawhen I open the elections22:48
* JayF looks for said ironic ptl email22:48
anteayahttp://lists.openstack.org/pipermail/openstack-dev/2014-April/031599.html22:49
anteayaJayF: ^22:49
NobodyCam:)22:49
JayFyeah my mail client is just a little slow22:50
JayFI think the IMAP server my host uses doesn't always properly notify unread count when server side filtering happeng22:50
JayF*happens22:50
*** max_lobur has quit IRC22:50
comstudprobably helps if I git add the file I added23:02
comstudhrm23:02
NobodyCamanteaya: thank you :)23:03
* NobodyCam need to do a quick walkies.. bbiafm23:05
NobodyCamcomstud: ya bot must have died :-p23:10
comstudyeah23:11
comstudi think i'm done for now23:11
NobodyCamya getting to EOD23:11
comstudi need a break at least23:11
comstud:)23:11
comstudbbl23:11
comstudtake care23:11
* comstud &23:12
NobodyCamTY comstud :)23:12
anteayanp23:15
*** derekh has joined #openstack-ironic23:23
*** stevehuang has quit IRC23:36
* NobodyCam calls it a day.. I'll be back in a while to check on things.. Night all23:42

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!