Thursday, 2015-09-17

crinklegreghaynes: yes00:00
greghaynescrinkle: the setup was eth2 -> eth2.25 -> br-vlan25 -> vetha -> vethb, we feed vethb to neutron.00:00
greghaynescrinkle: To make the pair its ip link add vetha type veth peer name vethb; brctl add br-vlan25 vetha00:01
crinklegreghaynes: I have ip link add veth1 type veth peer name veth2; brctl addif br-vlan25 veth100:01
greghaynesyea, that00:01
crinkleha00:01
greghaynesthen you have to ip link set dev veth* up00:01
crinkleya00:01
greghaynesand you should be good to go00:01
dims_thanks mordred for the https://review.openstack.org/#/c/224259/ - if anyone else with requirements karma please help as well, will unblock horizon (as it unblocks django_openstack_auth)00:02
*** nijaba has joined #openstack-infra00:02
*** nijaba has joined #openstack-infra00:02
mordredgreghaynes, Shrews: I sent a quick email to the ansible-devel mailing list warning people that there is a specific set of things they can do to trigger this and how to work around it00:03
*** nijaba_ has quit IRC00:03
mordred(tl;dr - just use auth_url intead of auth-url in your playbook, or just use a named cloud in clouds.yaml)00:03
*** Sukhdev has quit IRC00:04
*** thinrichs has joined #openstack-infra00:04
*** markvoelker has joined #openstack-infra00:06
*** aysyd has quit IRC00:07
mordredclarkb: we could catch keystoneauth1.exceptions.auth_plugins.MissingRequiredOptions and raise one that does the substitution we're wanting00:08
mordredalso00:08
mordredok. I'm going to sleep now. for realz00:08
greghaynesmordred: sure. we believe you.00:08
*** cloudtrainme has joined #openstack-infra00:08
* mordred has to get up in 4 hours to go give a keynote00:08
mordredMAYBE should have gone to bed a while ago :)00:09
*** Sukhdev has joined #openstack-infra00:10
clarkbgood night looks like nodepool is on its way to working00:10
clarkbso I will just have to approve that if we can get a +2 from someone else00:10
clarkbjeblair: ^00:10
greghaynesugh, so that race needs to get tracked down00:10
*** thinrichs has left #openstack-infra00:10
*** mdbooth has quit IRC00:10
clarkbgreghaynes: ya00:11
*** pfalleno1 has quit IRC00:11
*** stevemar has joined #openstack-infra00:11
*** markvoelker has quit IRC00:11
clarkbso one problem with the dsvm test is its going to ping keystoneauth to probably the previous version? how old is that release?00:12
clarkbits close enough of a test I suppose since unittests will do the right things00:12
*** yamamoto has joined #openstack-infra00:13
*** stevemar has quit IRC00:15
*** devlaps has quit IRC00:15
*** mdbooth has joined #openstack-infra00:16
*** glauco has joined #openstack-infra00:18
*** tsekiyama has quit IRC00:20
*** mtanino has quit IRC00:20
*** lawrancejing has quit IRC00:22
*** glauco has quit IRC00:22
*** Sukhdev has quit IRC00:23
*** rlandy has quit IRC00:23
*** dingyichen has joined #openstack-infra00:23
*** shashank_hegde has quit IRC00:27
greghaynesmordred: https://review.openstack.org/#/c/224368/00:27
greghaynescrinkle: is working for you?00:28
sridhar_ramclarkb: a question, perhaps noob one, on the devstack-gate (after browsing entries in projects.yaml). We currently have 'tox -e functional' to run the tests. Just wondering, is it guaranteed only this cmd will be invoked in the gate ?00:29
clarkbwell a lot more is done to setup devstack00:30
clarkbcan you expand on what you need or expect?00:31
*** achanda_ has quit IRC00:31
*** achanda has joined #openstack-infra00:33
sridhar_ramour need is fairly simple.. stack up devstack with our enable_plugin, cd /opt/stack/tacker, run 'tox -e functional' .. that's it00:35
sridhar_ramclarkb: also, I gather first we add the jobs to experimental queue before switching to the gate queue ?00:37
*** achanda has quit IRC00:38
ianwsridhar_ram : that seems like it would be fine done with gate hooks; many examples to follow00:39
ianwsridhar_ram: experimental maybe, non-voting probably; depends on the project & what they want00:39
claudiubhello, is there any infra person that can add me as a core for the os-win project that was just approved? https://review.openstack.org/#/c/209204/00:40
ianwsridhar_ram: but obviously committing a voting test that doesn't work kind of wedges you making forward progress00:40
*** abitha has quit IRC00:41
*** abitha has joined #openstack-infra00:42
sridhar_ramianw: I, perhaps naively, took 'experimental' as the way to run things that wouldn't block merges on test failure ... at least initially00:42
sridhar_ramI see pipeline: experimental .. now which attr control voting vs non-voting ?00:43
clarkbgreghaynes nodepool fix isnt a fix00:44
greghaynesclarkb: oh?00:44
greghaynesclatkb: I linked oscc fix00:44
greghaynesclarkb: that is another option00:44
clarkbya see the dsvm job00:45
clarkbI'm guessing 'auth' is important because oscc looks in 'auth'00:45
clarkbgreghaynes link?00:45
greghayneshttps://review.openstack.org/#/c/224368/00:45
greghaynesclarkb: yea, I was just pushing the believe button on not needing that sub dict00:46
kun_huangguys, anyone knows could PTL update gerrit label for his own project?00:46
clarkbgreghaynes should we do thst for all dicts?00:46
greghaynesclarkb: seems like either wait to merge that and release or we need to do our own munging in nodepool00:46
*** otter768 has joined #openstack-infra00:46
greghaynesclarkb: all?00:46
greghaynesclarkb: theres only one sub dict AFAIK00:46
*** abitha has quit IRC00:46
*** yamamoto has quit IRC00:47
clarkbuntio we add another00:47
greghaynesclarkb: youre saying type introspect instead of if == auth?00:47
clarkbkun_huang code review labels? can you explain what you would like to achieve?00:47
clarkbgreghaynes ya00:47
ianwsridhar_ram: experimental must be manually triggered "check experimental", while non-voting jobs are run always00:47
greghaynesclarkb: yea, thats fair, one sec00:47
*** sigmavirus24 is now known as sigmavirus24_awa00:48
*** dimtruck is now known as zz_dimtruck00:48
kun_huangclarkb: yep, like code-review -> vote00:48
clarkbkun_huang some projects have non default setups, what would you like to do?00:48
kun_huangjust want to know could PTL do it himself clarkb00:49
greghaynesclarkb: updated00:49
*** zz_dimtruck is now known as dimtruck00:49
clarkbno, you have to submut a change to your project acls byt tryibg to understand what you would like to do00:50
sridhar_ramianw: that helps! I'm tempted to go w/ non-voting gate jobs... something that gets run w/ every PS. the core team can manually honor them until we build confidence on the tests00:50
*** lawrancejing has joined #openstack-infra00:50
kun_huangclarkb: okay got it00:50
kun_huangclarkb: thanks :)00:50
*** otter768 has quit IRC00:51
clarkbgreghaynes use auth-url in your test not auth_url00:52
greghaynesclarkb: hah00:53
greghaynesclarkb: I did test the real thing, I just reworked the test afterwards ;)00:53
*** ozamiatin has joined #openstack-infra00:56
*** pvaneck has quit IRC00:56
*** tiswanso has joined #openstack-infra00:57
*** tiswanso has quit IRC00:58
*** tiswanso has joined #openstack-infra00:58
kevinbentonhi all, gate-grenade-dsvm-neutron is dying with this00:58
kevinbentonhttp://logs.openstack.org/37/224337/4/check/gate-grenade-dsvm-neutron/d0d019a/logs/grenade.sh.txt.gz#_2015-09-17_00_51_25_12400:58
*** kun_huang has left #openstack-infra00:58
kevinbentonblocking all neutron patches at least00:58
sridhar_ramianw: clarkb: for now I'll go with a gate-functional-dsvm-tacker job w/ voting: false in zuul/layout.yaml00:59
*** tiswanso_ has joined #openstack-infra00:59
*** dims_ has quit IRC01:01
*** tiswanso has quit IRC01:02
*** gildub has quit IRC01:02
openstackgerritMerged openstack-infra/irc-meetings: Change Horizon Drivers meeting ID  https://review.openstack.org/22426701:03
greghaynesclarkb: ok, we got one +2 on https://review.openstack.org/#/c/224368/01:03
greghaynesclarkb: just need to hunt down another core and release01:03
*** hdd has quit IRC01:03
*** gildub has joined #openstack-infra01:03
*** dimtruck is now known as zz_dimtruck01:04
*** plaqky17 has quit IRC01:05
*** bharathm has quit IRC01:05
*** geoffarn_ is now known as geoffarnoldX01:05
ruagairthanks jeblair. Yes I am ready for a storyboard db dump. Perhaps jhesketh can help out, as he's timezone friendly.01:06
*** markvoelker has joined #openstack-infra01:06
*** Daisy has joined #openstack-infra01:07
ianwclarkb: i think i know what's going on -- did you account for centos' "iptables-services" package?01:07
clarkbianw yes01:07
clarkbianw its in params.pp01:07
clarkbiptables is the puppet namvar but shouldnt be the package name01:08
clarkbbut maybe its broken01:08
ianwclarkb: ok, have to dig a bit, because i think maybe that's the issue01:08
ianw"systemctl enable missing-service" seems to give "access denied", rather than something helpful like "not around"01:08
*** pfallenop has joined #openstack-infra01:08
clarkbianw it did work with my daenon reload thing though01:09
ianwhmmm, this is a PITA.  my vm, that was failing with it the other dya, seems to have stopped replicating it01:10
ianwlet me do the dance to bring up a hpcloud node and see01:10
*** Daisy has quit IRC01:11
*** alexsyip has quit IRC01:14
*** shashank_hegde has joined #openstack-infra01:22
*** achanda has joined #openstack-infra01:22
clarkbgreghaynes I dont have oscc core oyherwise I would approve01:26
*** dims_ has joined #openstack-infra01:26
clarkbI guess we wait and dont restart nodepool :/01:26
tonybfungi: thanks for following up I got to the bottom of it.01:26
greghaynesclarkb: yea01:26
*** larainema has quit IRC01:27
*** yamahata has quit IRC01:27
*** larainema has joined #openstack-infra01:27
*** larainema has quit IRC01:28
*** EricGonczer_ has joined #openstack-infra01:29
*** larainema has joined #openstack-infra01:29
*** larainema has quit IRC01:30
*** larainema has joined #openstack-infra01:31
*** zhenguo has joined #openstack-infra01:31
*** fitoduarte has quit IRC01:33
*** thorst has quit IRC01:33
*** yolanda has quit IRC01:36
*** sridhar_ram has quit IRC01:36
*** FallenPegasus is now known as MarkAtwood01:39
*** xyang1 has quit IRC01:39
*** achanda has quit IRC01:41
*** ozamiatin has quit IRC01:44
*** shashank_hegde has quit IRC01:45
*** lawrance_ has joined #openstack-infra01:46
*** chlong has joined #openstack-infra01:47
*** Zhongjun has quit IRC01:47
*** ashleigh has quit IRC01:47
*** lawrancejing has quit IRC01:49
*** edmondsw has quit IRC01:52
*** chlong_ has joined #openstack-infra01:53
*** dingyichen has quit IRC01:55
*** dingyichen has joined #openstack-infra01:56
*** larainema has quit IRC01:57
*** larainema has joined #openstack-infra01:58
*** bpokorny has quit IRC01:58
*** MarkAtwood has quit IRC01:59
*** gildub has quit IRC02:00
*** jerryz has quit IRC02:01
*** r-daneel has quit IRC02:02
*** ybathia has quit IRC02:04
*** EricGonczer_ has quit IRC02:10
*** unicell has quit IRC02:11
*** stevemar has joined #openstack-infra02:12
*** zeih has joined #openstack-infra02:14
*** rick-chen has joined #openstack-infra02:16
*** stevemar has quit IRC02:16
*** salv-orlando has joined #openstack-infra02:18
*** yamahata has joined #openstack-infra02:18
*** zeih has quit IRC02:18
*** yamamoto has joined #openstack-infra02:19
*** zz_dimtruck is now known as dimtruck02:19
rick-chenDoes anyone know how to trace CI result unable to post on the git review window?02:20
*** r1chardj0n3s has joined #openstack-infra02:23
*** salv-orlando has quit IRC02:23
r1chardj0n3slifeless: we're still seeing devstack fail in django-openstack-auth after your fix https://review.openstack.org/#/c/222478/02:24
*** EricGonczer_ has joined #openstack-infra02:24
*** ddieterly has joined #openstack-infra02:25
*** kengo_sakai has joined #openstack-infra02:28
*** yamamoto has quit IRC02:29
*** EricGonczer_ has quit IRC02:30
*** rlandy has joined #openstack-infra02:30
*** david-lyle has joined #openstack-infra02:34
*** ddieterl_ has joined #openstack-infra02:35
*** ddieterly has quit IRC02:35
*** hdd has joined #openstack-infra02:36
jheskethruagair: not at the moment I'm not ;-)02:39
*** achanda has joined #openstack-infra02:41
ruagairah, so sneaky :-D02:41
ruagairThen I'll make is a general call for someone who has root to grab me a dump of storyboard so I can test the phabricator import :-D02:42
*** huanxie has joined #openstack-infra02:43
*** harlowja_at_home has joined #openstack-infra02:43
*** gildub has joined #openstack-infra02:45
*** achanda has quit IRC02:45
*** otter768 has joined #openstack-infra02:47
*** earlephilhower has quit IRC02:47
*** chlong_ has quit IRC02:50
openstackgerritEmilien Macchi proposed openstack-infra/project-config: zuul/layout: puppet openstack updates  https://review.openstack.org/22439702:51
EmilienMcrinkle: ^02:52
*** otter768 has quit IRC02:52
EmilienMmordred: have you tried to run compute tests against cloudwatt ?02:52
ianwclarkb: ok, new theory.  i think updating the selinux rules first might be the solution.  the daemon-reload doesn't seeem tow work, as we've noticed02:56
*** dpaterson has quit IRC03:03
*** achanda has joined #openstack-infra03:03
clarkbianw I think the package needs to reload03:06
clarkbwe get a new unit file and until the reload happens systemd has no idea03:06
*** MarkAtwood has joined #openstack-infra03:07
openstackgerritEmilien Macchi proposed openstack-infra/project-config: zuul/layout: puppet openstack updates  https://review.openstack.org/22439703:09
ianwclarkb: the thing is, the iptables-services should be doing that, from the spec file03:10
EmilienMhello, I know if anyone could review zuul/layout change for puppet: https://review.openstack.org/#/c/224397/ - thanks and good night/day03:10
*** btully has quit IRC03:11
StevenKpleia2: I'm rebuilding my pretend proposal slave, then I'm going to dig into the glossary fun and games03:13
pleia2StevenK: thank you, I got tired03:14
*** yamamoto has joined #openstack-infra03:15
*** coolsvap is now known as coolsvap|holiday03:17
*** Shrews has quit IRC03:19
*** Shrews has joined #openstack-infra03:19
*** glauco has joined #openstack-infra03:19
ianwclarkb: actually, what happens is the .spec file calls "systemctl preset iptables" which is the same as "enable" which handles the equivalent of the reload (http://www.freedesktop.org/software/systemd/man/systemctl.html)03:20
*** r1chardj0n3s has quit IRC03:20
*** r1chardj0n3s has joined #openstack-infra03:21
*** jyuso1 has joined #openstack-infra03:22
*** purp has quit IRC03:22
*** purp has joined #openstack-infra03:22
*** glauco has quit IRC03:24
*** geoffarnoldX is now known as geoffarn_03:26
openstackgerritfumihiko kakuma proposed openstack-infra/project-config: Add job for the ovs mechanism driver with native mode  https://review.openstack.org/22114303:28
clarkbthe man page just says its the same as enable or disable03:28
clarkbI dont see where it does a reload03:28
clarkbenable != reload03:29
*** cloudtrainme has quit IRC03:29
ianwclarkb: under "enable" -> "After the symlinks have been created, the systemd configuration is reloaded (in a way that is equivalent to daemon-reload) to ensure the changes are taken into account immediately."03:29
*** bpokorny has joined #openstack-infra03:30
ianwit has to work, because that's how the generic rpm macro does it, which works for everything else03:30
clarkbmy experinece is that is not sufficient on centos703:30
*** mrmartin has joined #openstack-infra03:30
clarkbI had to manually reload all the things03:30
clarkbenable would conplain that the service is unknown, reload then I could enable03:31
clarkbbut maybe thats the selinux issue?03:31
ianwclarkb: https://github.com/systemd/systemd/blob/master/src/core/macros.systemd.in#L39 <- systemd_post macro , which is what every package calls03:31
*** tlian has quit IRC03:32
ianwso iptables-services installs itself, then calls systemd_post macro (systemctl preset iptables), which fails due to the selinux error03:32
ianwthat actually gets hidden by rpm, i guess that script doesn't run with "-e"03:33
clarkbis selinux problem related to the symlinking?03:33
ianwbut then we notice when we do "systemctl enable iptables"03:33
*** claudiub has quit IRC03:33
clarkbhrm must not he because explicit reload later works03:34
ianwclarkb: i am wondering about the split of iptables/iptables-services and if that affects things03:34
clarkbwhich implies symlinks are rhere03:34
ianwwhy a reload of systemd *after* iptables is installed i'm not 100% sure, yet03:34
clarkboh is it presetting iptables-services?03:34
clarkbbut service name is actually iptables?03:34
*** achanda has quit IRC03:35
*** achanda has joined #openstack-infra03:35
*** harlowja_at_home has quit IRC03:35
ianw%postun services03:35
ianw/sbin/ldconfig03:35
ianw%systemd_postun_with_restart iptables.service ip6tables.service03:35
ianwno03:35
clarkbI dont know what %{?*} evaluates to03:36
ianwyou can also see that from the avc denial in https://bugzilla.redhat.com/show_bug.cgi?id=126174703:36
openstackbugzilla.redhat.com bug 1261747 in systemd "Upgrade of systemd leaves service enablement failing with selinux error" [Unspecified,New] - Assigned to systemd-maint03:36
ianw^ that's the wrong paste above  (postun), but yeah, it specifies the names correctly03:37
clarkbI see, so the scontext is unconfined but it wants it to be system_u03:37
clarkbI wobder if yum needs to be part of the init context03:38
clarkbwould be good to see if other packages fail this way too03:38
clarkbbut I dont know enough about yum/rpm to know if that is really a good way to approach that03:39
*** tlian has joined #openstack-infra03:40
ianwi feel like it might be a bug in the selinux targeted policy that is maybe fixed, but i can't pinpoint it yet03:40
clarkbor maybe systemctl needs to never inherit context and have a rule for very specific needs?03:40
*** olaph has quit IRC03:41
clarkbianw maybe fixed? because rax is ok?03:41
*** ivar-laz_ has joined #openstack-infra03:41
ianwclarkb: maybe, because it might have a higher version03:41
ianwthe fix might be upgrade selinux-policy , restart systemd , install iptables-services as usual03:42
*** rlandy has quit IRC03:43
*** ivar-lazzaro has quit IRC03:44
*** vmbrasseur is now known as vmbrasseur_BIAB03:44
*** vmbrasse- is now known as vmbrasseur03:44
*** geoffarn_ has quit IRC03:44
*** ivar-laz_ has quit IRC03:45
*** boris-42 has quit IRC03:50
*** sdake_ has joined #openstack-infra03:51
*** boris-42 has joined #openstack-infra03:51
greghaynesclarkb: ok, merged. Now we just need a release...03:51
clarkbgreghaynes :/ its time to sleep not make releases03:52
greghaynesclarkb: for NA, yes03:52
clarkbmaybe mordred will wake in a few hours03:52
greghaynesyea03:53
*** harlowja_at_home has joined #openstack-infra03:53
*** sdake has quit IRC03:54
rm_workanyone know why this might happen?03:55
rm_work2015-09-17 01:11:00.440 | + git fetch http://zm03.openstack.org/p/openstack-infra/devstack-gate refs/zuul/master/Za8018611f6f441aaa4e2dbb328a07c3c03:55
rm_work2015-09-17 01:11:00.673 | fatal: Couldn't find remote ref refs/zuul/master/Za8018611f6f441aaa4e2dbb328a07c3c03:55
rm_workin devstack setup03:55
rm_workhttp://logs.openstack.org/65/223265/9/check/gate-grenade-dsvm-neutron/2ef176b/logs/devstack-gate-setup-workspace-new.txt.gz03:55
rm_workand http://logs.openstack.org/45/224345/2/check/gate-grenade-dsvm-neutron/bdf49f2/logs/devstack-gate-setup-workspace-new.txt.gz03:56
rm_workboth03:56
rm_worknot sure why that would be a thing03:56
*** hdd has quit IRC03:58
ianwi think that's an expected failure03:58
rm_workerm03:58
rm_workhmm03:58
ianwuselessly, i can't remember why03:58
rm_workok so maybe not the problem03:59
anteayajeblair: http://23.253.49.144/ looks good and about the look I got as far as css03:59
* rm_work keeps looking03:59
anteayajeblair: doing the rest by hand for this round sounds reasonable to me03:59
rm_workoh i guess for a grenade test i look in grenade.sh03:59
rm_workhmm03:59
rm_workok looks like 2015-09-17 01:21:39.403 | pkg_resources.ContextualVersionConflict: (Babel 2.0 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('Babel<=1.3,>=1.3'), set(['oslo.utils']))03:59
rm_workpart of keystone?04:00
rm_workduring 2015-09-17 01:21:39.146 | + /usr/local/bin/keystone-manage db_sync04:00
*** ddieterl_ has quit IRC04:00
rm_worksorry, debugging out loud04:00
greghaynesmordred: Youre probably keynoting right now, but when you return we could use a new release of oscc :)04:02
rm_workis it that someone merged a requirements update and someone didn't yet?04:03
rm_workcould that cause the pkg_resources.ContextualVersionConflict issue?04:03
*** unicell has joined #openstack-infra04:03
*** yamahata has quit IRC04:03
*** unicell has quit IRC04:03
*** unicell has joined #openstack-infra04:03
*** geoffarnold has joined #openstack-infra04:04
greghaynesrm_work: That says there is Babel 2.0 installed but Babel 1.3 is specified in requirements04:08
*** tiswanso_ has quit IRC04:10
Nakatorm_work: Looks like keystonemiddleware that is being installed gets olso.utils 1.4.1 which requires "Babel<=1.3,>=1.3" and 2.0 has been installed already by keystonemiddleware 1.5.2.04:11
*** larainema has quit IRC04:11
greghaynesYerp04:12
*** stevemar has joined #openstack-infra04:12
rm_workhm04:13
rm_workawesome04:14
*** fawadkhaliq has joined #openstack-infra04:15
*** zeih has joined #openstack-infra04:15
tonybrm_work: let me have a look at that, it could be my fault ....04:15
*** chlong has quit IRC04:16
*** stevemar has quit IRC04:16
tonybrm_work: what's the review url?04:16
*** hdd has joined #openstack-infra04:16
*** reed_ has joined #openstack-infra04:16
*** reed_ has quit IRC04:18
*** zeih has quit IRC04:19
rm_workhttps://review.openstack.org/#/c/224345/2 is one04:19
rm_worki just rebased the review below it, maybe it randomly will fix itself? :P04:19
rm_workwouldn't that be nice04:19
*** harlowja_at_home has quit IRC04:19
greghayneswe rebase before running tests and fail if we cant, so unlikely04:20
rm_workah :/04:20
rm_workTIL04:20
*** olaph has joined #openstack-infra04:21
NakatoFairly widespread http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwicGtnX3Jlc291cmNlcy5Db250ZXh0dWFsVmVyc2lvbkNvbmZsaWN0OiAoQmFiZWwgMi4wICgvdXNyL2xvY2FsL2xpYi9weXRob24yLjcvZGlzdC1wYWNrYWdlcyksIFJlcXVpcmVtZW50LnBhcnNlKCdCYWJlbDw9MS4zLD49MS4zJyksIHNldChbJ29zbG8udXRpbHMnXSkpXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjQzMjAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQ0MjQ2MzY2ODM4MX0=04:21
rm_worknice tool04:22
*** stevemar has joined #openstack-infra04:23
greghaynesthis sort of thing usually happens if we need to pin something in our stable branch04:24
greghaynesonly half here so cant fully debug though04:24
*** amotoki has joined #openstack-infra04:25
tonybrm_work: No I think it'm my fault oslo.utils for juno came out about 4 hours ago.   too much of a coincidence04:25
rm_workhmm k04:26
StevenKpleia2: Still around?04:26
greghaynesyea, so that sounds like the typical case where this happens04:26
Nakatotonyb: You just tagged 1.4.1 if I'm seing right.  Which switched requirements.txt from Babel>=1.3 to Babel<=1.3,>=1.3.  So it looks like that might be it.04:26
tonybNakato: correct.04:27
*** larainema has joined #openstack-infra04:27
lifelessr1chardj0n3s: my fix?04:31
lifelessr1chardj0n3s: https://review.openstack.org/#/c/224259/ is the fix04:31
lifelessr1chardj0n3s: and as you can see it hasn't landed yt04:31
*** unicell1 has joined #openstack-infra04:32
r1chardj0n3soh, that's weird, someone said it had merged, and for some reason when I looked at it I thought it had. sorry, braino04:32
lifelessr1chardj0n3s: de nada04:33
*** chlong has joined #openstack-infra04:33
*** unicell has quit IRC04:33
tonybGah so oslo utils was supposed to the for juno only and it's being used in kilo and hence the big bang04:37
*** dims_ has quit IRC04:38
*** achanda has quit IRC04:39
tonybSo I think the quick fix is the blacklist 1.4.1 from kilo and then qorkout how to fix the overlapping ranges.04:41
*** fedexo has joined #openstack-infra04:42
*** dims_ has joined #openstack-infra04:43
*** bpokorny has quit IRC04:44
*** dims_ has quit IRC04:44
*** pcrews has quit IRC04:46
*** sdake has joined #openstack-infra04:46
*** bpokorny has joined #openstack-infra04:46
*** chlong has quit IRC04:48
*** bpokorny has quit IRC04:48
*** sdake_ has quit IRC04:49
*** salv-orlando has joined #openstack-infra04:50
*** ivar-lazzaro has joined #openstack-infra04:50
*** unicell1 has quit IRC04:51
*** unicell has joined #openstack-infra04:51
*** lawrance_ has quit IRC04:53
*** MarkAtwood has quit IRC04:56
rm_workok cool, so there exists a solution :)04:56
*** pcrews has joined #openstack-infra04:56
*** salv-orlando has quit IRC04:57
*** nadya has joined #openstack-infra04:57
*** unicell has quit IRC04:58
*** yfried_ has quit IRC04:58
*** nadya has quit IRC04:59
*** rkukura_ has joined #openstack-infra04:59
tonybrm_work: Umm yeah, I wish it were a quicker solution.04:59
*** rkukura has quit IRC05:00
*** rkukura_ is now known as rkukura05:00
*** ivar-lazzaro has quit IRC05:00
*** ddieterly has joined #openstack-infra05:01
*** ivar-lazzaro has joined #openstack-infra05:01
*** nadya has joined #openstack-infra05:02
*** dimtruck is now known as zz_dimtruck05:03
*** MarkAtwood has joined #openstack-infra05:07
openstackgerritTony Breeds proposed openstack-infra/elastic-recheck: Add signatire for 1496678  https://review.openstack.org/22442505:08
*** nadya has quit IRC05:09
*** btully has joined #openstack-infra05:12
*** hdd has quit IRC05:19
*** achanda has joined #openstack-infra05:20
*** hrou has quit IRC05:20
*** unicell has joined #openstack-infra05:28
openstackgerritTony Breeds proposed openstack/requirements: Ban oslo.utils 1.4.1 from kilo  https://review.openstack.org/22442905:28
*** stevemar has quit IRC05:28
*** stevemar has joined #openstack-infra05:29
*** geoffarnold is now known as geoffarnoldX05:29
mordredgreghaynes, clarkb: occ released05:35
*** rick-chen has quit IRC05:36
openstackgerritTimo Tijhof proposed openstack-infra/zuul: Pass change and pipeline to the status_url  https://review.openstack.org/8690005:39
*** fedexo has quit IRC05:42
*** hdd has joined #openstack-infra05:44
greghaynesmordred: awesome, ty05:44
openstackgerritTimo Tijhof proposed openstack-infra/zuul: Pass change and pipeline to the status_url  https://review.openstack.org/8690005:44
*** dims_ has joined #openstack-infra05:45
openstackgerritTimo Tijhof proposed openstack-infra/zuul: Pass change and pipeline to the status_url  https://review.openstack.org/8690005:45
*** zeih has joined #openstack-infra05:45
*** hdd has quit IRC05:47
mordredgreghaynes: thanks for th epatch!05:47
openstackgerritIan Wienand proposed openstack-infra/system-config: Systemd+selinux vs iptables on centos7, round 2  https://review.openstack.org/22443305:50
*** zeih has quit IRC05:50
ianwclarkb: ^ for later.  hopefully we can pinpoint a cause in the rhbz, but my testing shows this works.  I *think* with the old change I had inadvertently run a daemon reload after the iptables install, convincing myself it worked when it didn't05:52
*** ivar-lazzaro has quit IRC05:52
*** dims_ has quit IRC05:53
*** ivar-lazzaro has joined #openstack-infra05:53
*** Sukhdev has joined #openstack-infra05:54
*** cloudtrainme has joined #openstack-infra05:54
*** cloudtrainme has quit IRC05:54
*** cloudtrainme has joined #openstack-infra05:55
*** che-arne has joined #openstack-infra05:56
*** asettle has quit IRC05:57
*** bexelbie_ has joined #openstack-infra06:00
*** bexelbie_ is now known as bexelbie06:01
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/22443706:02
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/22115706:07
openstackgerritAndrey Pavlov proposed openstack-infra/project-config: ec2api job: enable voting of py34 for master branch  https://review.openstack.org/22444106:07
*** lawrancejing has joined #openstack-infra06:07
*** aeng has quit IRC06:09
*** Daisy has joined #openstack-infra06:12
*** bharathm has joined #openstack-infra06:15
*** armax has quit IRC06:15
*** chlong has joined #openstack-infra06:18
*** yfried_ has joined #openstack-infra06:19
*** yfried__ has joined #openstack-infra06:23
*** yfried_ has quit IRC06:24
*** yamahata has joined #openstack-infra06:28
*** balajin has quit IRC06:28
*** MarkAtwood has quit IRC06:29
*** nadya has joined #openstack-infra06:29
*** harlowja has quit IRC06:29
*** rkukura_ has joined #openstack-infra06:29
*** MarkAtwood has joined #openstack-infra06:29
*** rkukura has quit IRC06:29
*** rkukura_ is now known as rkukura06:29
*** zhenguo has quit IRC06:30
*** zhenguo has joined #openstack-infra06:32
*** nadya has quit IRC06:33
*** abregman has joined #openstack-infra06:35
*** balajin has joined #openstack-infra06:36
*** salv-orlando has joined #openstack-infra06:37
*** yfried__ is now known as yfried|afk06:37
*** scheuran has joined #openstack-infra06:38
*** yfried|afk is now known as yfried__06:40
*** salv-orlando has quit IRC06:43
*** MarkAtwood has quit IRC06:43
*** zeih has joined #openstack-infra06:46
*** otter768 has joined #openstack-infra06:49
ttxjeblair: yeah, if I have SSH access to that one I can probably make it go the last mile now. I'll note down what I had to do to "finalize" it so we can improve the puppet recipe later06:50
*** zeih has quit IRC06:51
*** dingyichen has quit IRC06:52
*** markvoelker has quit IRC06:53
*** otter768 has quit IRC06:54
*** bexelbie has quit IRC06:55
*** dizquierdo has joined #openstack-infra06:55
*** e0ne has joined #openstack-infra06:56
*** claudiub has joined #openstack-infra06:58
openstackgerritKevin Carter proposed openstack-infra/project-config: Added periodic job for OpenStack-Ansible  https://review.openstack.org/22421506:59
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/22443707:02
*** e0ne has quit IRC07:03
*** ivar-lazzaro has quit IRC07:05
*** peristeri has joined #openstack-infra07:08
*** HeOS_ has quit IRC07:09
*** stevemar has quit IRC07:10
*** ildikov has quit IRC07:10
*** Sukhdev has quit IRC07:11
*** huanxie has quit IRC07:14
*** fawadkhaliq has quit IRC07:17
*** arxcruz has joined #openstack-infra07:18
*** salv-orlando has joined #openstack-infra07:18
*** Ala has joined #openstack-infra07:19
*** markus_z has joined #openstack-infra07:19
*** glauco has joined #openstack-infra07:21
*** s0nea has quit IRC07:21
*** markus_z has quit IRC07:24
*** shardy has joined #openstack-infra07:25
*** glauco has quit IRC07:26
*** yamahata has quit IRC07:26
openstackgerritMerged openstack-infra/tripleo-ci: Use a specific fedora mirror on the jenkins node  https://review.openstack.org/22386807:26
*** dtantsur|afk is now known as dtantsur07:27
openstackgerrityolanda.robla proposed openstack-infra/puppet-jenkins: Allow to specify a different jenkins default  https://review.openstack.org/21148107:28
*** bexelbie_ has joined #openstack-infra07:29
*** yolanda has joined #openstack-infra07:30
*** claudiub has quit IRC07:31
*** alex_xu_ is now known as alex_xu07:32
*** Hal has joined #openstack-infra07:32
*** yfried has joined #openstack-infra07:34
*** hashar has joined #openstack-infra07:35
*** larainema has quit IRC07:36
*** yfried__ has quit IRC07:36
*** larainema has joined #openstack-infra07:36
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Use a specific mirror for CentOS builds  https://review.openstack.org/22400007:37
openstackgerrityolanda.robla proposed openstack-infra/puppet-jenkins: Allow to specify a different jenkins default  https://review.openstack.org/21148107:38
*** achanda has quit IRC07:38
*** larainema has quit IRC07:41
*** larainema has joined #openstack-infra07:42
*** jlanoux has joined #openstack-infra07:44
*** ildikov has joined #openstack-infra07:45
*** shardy_ has joined #openstack-infra07:47
*** shardy has quit IRC07:47
*** achanda has joined #openstack-infra07:47
*** larainema has quit IRC07:49
*** larainema has joined #openstack-infra07:49
*** achanda has quit IRC07:50
*** dims_ has joined #openstack-infra07:51
*** shardy_ has quit IRC07:51
*** fawadkhaliq has joined #openstack-infra07:51
*** shardy has joined #openstack-infra07:52
openstackgerrityolanda.robla proposed openstack-infra/puppet-jenkins: Properly setup of jenkins keys  https://review.openstack.org/21519707:52
*** svasheka has quit IRC07:54
*** yfried has quit IRC07:54
*** markvoelker has joined #openstack-infra07:54
*** gildub has quit IRC07:55
*** yfried has joined #openstack-infra07:55
*** Daisy_ has joined #openstack-infra07:56
*** dims_ has quit IRC07:57
*** ifarkas has joined #openstack-infra07:59
*** markvoelker has quit IRC07:59
*** mpaolino has joined #openstack-infra08:01
*** svasheka has joined #openstack-infra08:01
*** Daisy_ has quit IRC08:01
-openstackstatus- NOTICE: Gate is currently stuck, failing grenade upgrade tests due the release of oslo.utils 1.4.1 for Juno.08:01
hrubijhesketh: hi, did you have time to check my comment in https://review.openstack.org/#/c/121528/35/zuul/scheduler.py ?08:03
*** s0nea has joined #openstack-infra08:03
*** btully has quit IRC08:03
openstackgerritJan Klare proposed openstack-infra/project-config: move 'gate-{name}-chef-rake-integration' job  https://review.openstack.org/19418108:03
*** yolanda has quit IRC08:04
*** sflanigan has quit IRC08:05
*** fhubik has joined #openstack-infra08:10
lifelesstonyb: ttx: ok so08:10
tonyblifeless: I see08:10
lifelessKilo has no constraints protection which is why its broken08:11
tonyblifeless: I didn't know about : gate-oslo.utils-requirements which fails as you said it would08:11
lifelessI *thought* the proposed plan included the kilo caps, but clearly not. To fix it we'd need to update kilo to make sure it doesn't get the juno release08:11
lifelessassuming thats even possible08:11
tonyblifeless: Well the quickest fix would be to ban 1.4.1, which I had out there but I abandoned it in favor of this plan that wont work08:12
tonybthat will make it nearly impossible to do further release of 1.4 for kilo though (IIUC)08:13
ttxbah, one problem at a time08:14
tonybttx: :)08:15
lifelessand this is why branches of libraries.. gets off of hobby horse08:15
tonyblifeless: :)08:15
*** larainema has quit IRC08:16
lifelesshttps://etherpad.openstack.org/p/branchless-clients-and-libs08:16
lifelessanyhoo08:16
tonyblifeless: so should I restore my ban 1.4.1 in kilo review?08:16
*** larainema has joined #openstack-infra08:17
lifelesstonyb: I don't know what that review looks like08:17
lifelesstonyb: so I can't answer that08:17
tonyblifeless: https://review.openstack.org/#/c/224429/08:17
*** markus_z has joined #openstack-infra08:18
tonyblifeless: IIUC banning 1.4.1 from juno will mean it gets 1.4.0 (again) and juno still gets 1.4.1 ....08:18
openstackgerritDmitry Tantsur proposed openstack-infra/project-config: Make gate-python-ironic-inspector-client-dsvm voting  https://review.openstack.org/22447608:19
*** jistr has joined #openstack-infra08:20
tonyblifeless: but that'll have a cascade effect where we need to get g-r updates into every library that uses oslo.utils wont it?08:20
lifelessyes, unabandon that08:20
lifelessyes08:20
ttxlooks like that can't hurt08:20
lifelesswelcome to why I'm pushing on the stuff I'm pushing on08:20
lifelesstonyb: would you like to help me? :)08:20
tonyblifeless: we can talk about it in Tokyo08:21
tonyblifeless: I'm not agreeing to anything right now ;P08:21
tonyblifeless: restored that it's in the check queue, bonus is it already has a +208:23
*** Daisy has quit IRC08:23
openstackgerritBob Ball proposed openstack-infra/project-config: Add fuel-plugin-xenserver project  https://review.openstack.org/22415108:26
tonybooo so much red in the gate.08:28
ttxtonyb: approving08:30
tonybttx: Thanks :)08:30
ttxgating now08:30
ttx(since it already passed tests)08:30
tonybttx:After it merges will the g-r sync bot do it's think or does that need a kick08:31
lifelessit's event driven08:32
ttxtonyb: I think it's running daily, so we might want to push down the syncs we need08:32
ttxlifeless: is it ?08:32
lifelessttx: yes08:32
lifelessttx: the constraints generator is daily08:32
lifelessttx: this is not that08:32
ttxoh, cool.08:32
*** derekh has joined #openstack-infra08:33
*** btully has joined #openstack-infra08:34
*** ozamiatin has joined #openstack-infra08:35
*** ihrachys has joined #openstack-infra08:36
*** btully has quit IRC08:38
*** ams___ is now known as ams_08:40
*** ams_ has joined #openstack-infra08:40
*** ams_ has joined #openstack-infra08:40
*** lawrancejing has quit IRC08:41
*** lawrancejing has joined #openstack-infra08:41
*** bharathm has quit IRC08:43
odyssey4medoes the oslo.utils 1.4.1 library only affect kilo?08:43
*** romcheg has joined #openstack-infra08:44
tonybodyssey4me: yes.08:45
tonybodyssey4me: master has >=2.0.0 so it's only a juno/kilo thing and juno is fine with 1.4.1 so that leaves only kilo08:46
*** ozamiatin has quit IRC08:46
tonybttx, lifeless: I need to leave.  I'll be back in about 2 - 2.5 hours to try and pick up where I left off.08:47
odyssey4meok, cool - waiting for the patch to merge so that I can update the openstack-ansible sha's to include the restriction08:47
tonybttx, lifeless: thanks for you help.08:47
*** zeih has joined #openstack-infra08:47
ttxtonyb: thanks for staying around to fix it !08:47
*** nadya has joined #openstack-infra08:47
*** HeOS_ has joined #openstack-infra08:47
tonybttx: np.08:47
*** soren has joined #openstack-infra08:48
*** ozamiatin has joined #openstack-infra08:49
*** teran has quit IRC08:49
openstackgerritDoron Chen proposed openstack-infra/project-config: Adding a functional test to the storlets project  https://review.openstack.org/22013908:49
*** hichihara has quit IRC08:50
*** zeih has quit IRC08:51
*** ozamiatin has quit IRC08:52
openstackgerritMehdi Abaakouk (sileht) proposed openstack/requirements: Raise jsonpath-rw-ext version  https://review.openstack.org/22449408:54
*** alexpilotti has joined #openstack-infra08:55
*** yfried is now known as yfried|afk08:56
openstackgerritMehdi Abaakouk (sileht) proposed openstack/requirements: Raise jsonpath-rw-ext version  https://review.openstack.org/22449408:56
*** oomichi has joined #openstack-infra08:58
openstackgerritMichal Rostecki proposed openstack-infra/project-config: Fix jobs names for Puppet 4.0  https://review.openstack.org/22449608:59
*** jordanP has joined #openstack-infra08:59
*** nihilifer has joined #openstack-infra09:00
openstackgerritMerged openstack-infra/project-config: Add PyPi jobs to yaql  https://review.openstack.org/22417609:01
*** madong_ has joined #openstack-infra09:02
*** larainema has quit IRC09:02
*** rick-chen has joined #openstack-infra09:03
openstackgerritDerek Higgins proposed openstack/diskimage-builder: Download a compressed centos cloud images  https://review.openstack.org/22415509:04
ttxheh weird. The requirements change is in both the check and the gate queue.09:05
ttxI bet that's a sideeffect of the unabandon09:05
rick-chenfungi: ping09:06
*** rick-chen_ has joined #openstack-infra09:09
*** rick-chen has quit IRC09:11
ttxSergeyLukjanov: around for a quick release status update ?09:17
openstackgerritMerged openstack/requirements: Ban oslo.utils 1.4.1 from kilo  https://review.openstack.org/22442909:17
ttxoops. wrong channel09:17
ttxlifeless: the requirements change just merged. Anything else we need to do to unblock this ?09:18
lifelessttx: I don't know. Try a recheck on a master patch09:19
*** yfried|afk is now known as yfried09:20
ttxon it09:20
*** ociuhandu has quit IRC09:25
*** ozamiatin has joined #openstack-infra09:26
*** pelix has joined #openstack-infra09:27
*** lawrancejing has quit IRC09:28
openstackgerritMehdi Abaakouk (sileht) proposed openstack/requirements: Raise jsonpath-rw-ext version  https://review.openstack.org/22449409:28
*** peristeri has quit IRC09:29
*** yfried is now known as yfried|afk09:30
*** fhubik has quit IRC09:33
*** fhubik has joined #openstack-infra09:33
*** Daisy has joined #openstack-infra09:39
*** fhubik_ has joined #openstack-infra09:39
openstackgerritMonty Taylor proposed openstack-infra/shade: Use the get_auth function from occ  https://review.openstack.org/22112609:42
openstackgerritMonty Taylor proposed openstack-infra/shade: Add wrapper for heat's get_template_contents  https://review.openstack.org/22292909:42
openstackgerritMonty Taylor proposed openstack-infra/shade: Abstract out the name of the name key  https://review.openstack.org/22293009:42
openstackgerritMonty Taylor proposed openstack-infra/shade: Add heatclient support  https://review.openstack.org/22285609:42
openstackgerritMonty Taylor proposed openstack-infra/shade: Start using keystoneauth for keystone sessions  https://review.openstack.org/18317609:42
openstackgerritMonty Taylor proposed openstack-infra/shade: Add heat support  https://review.openstack.org/15190009:42
*** fawadkhaliq has quit IRC09:42
*** fawadkhaliq has joined #openstack-infra09:43
*** Daisy has quit IRC09:43
*** fhubik is now known as fhubik_brb09:43
openstackgerritMateusz Matuszkowiak proposed openstack-infra/project-config: Added new Openstack repository for Fuel Zabbix Agents integration  https://review.openstack.org/22125909:43
*** teran has joined #openstack-infra09:45
*** fawadkhaliq has quit IRC09:47
*** e0ne has joined #openstack-infra09:51
*** markvoelker has joined #openstack-infra09:55
*** rick-chen_ has quit IRC09:56
*** cdent has joined #openstack-infra09:56
*** vgridnev has joined #openstack-infra09:57
*** dims_ has joined #openstack-infra09:59
*** markvoelker has quit IRC10:00
*** e0ne has quit IRC10:00
*** fawadkhaliq has joined #openstack-infra10:02
*** cfarquhar has quit IRC10:06
*** cfarquhar has joined #openstack-infra10:08
*** cfarquhar has quit IRC10:08
*** cfarquhar has joined #openstack-infra10:08
*** fhubik_ has quit IRC10:10
*** fhubik_brb has quit IRC10:10
ttxLooks like we have some grenade jobs passing now, so we may be unblocked10:10
*** yfried|afk is now known as yfried10:10
*** e0ne has joined #openstack-infra10:11
*** fhubik has joined #openstack-infra10:11
*** mrmartin has quit IRC10:11
*** fhubik has quit IRC10:13
*** fhubik has joined #openstack-infra10:13
ttx#status notice Gate back to normal, thanks to the backlisting of the problematic version10:14
openstackstatusttx: sending notice10:14
-openstackstatus- NOTICE: Gate back to normal, thanks to the backlisting of the problematic version10:15
openstackgerritAntoine Musso proposed openstack/diskimage-builder: Prettify 'Caches and offline mode' documentation  https://review.openstack.org/22455410:16
openstackstatusttx: finished sending notice10:16
*** kengo_sakai has quit IRC10:17
openstackgerritMerged openstack/diskimage-builder: Don't create a centos yum repository  https://review.openstack.org/22399610:23
*** peristeri has joined #openstack-infra10:24
*** IvanKliuk has joined #openstack-infra10:25
*** yfried is now known as yfried|afk10:26
IvanKliukHello!10:27
*** salv-orlando has quit IRC10:27
dims_thanks ttx and tonyb10:28
IvanKliukColleagues, can you please review https://review.openstack.org/#/c/223980/ ?10:29
*** zeih has joined #openstack-infra10:33
*** soren has quit IRC10:35
*** zeih has quit IRC10:37
*** yfried|afk is now known as yfried10:37
*** salv-orlando has joined #openstack-infra10:38
*** pc_m has joined #openstack-infra10:39
*** zhenguo has quit IRC10:40
*** lawrancejing has joined #openstack-infra10:46
*** mpaolino has quit IRC10:47
*** ZZelle has quit IRC10:50
*** ZZelle has joined #openstack-infra10:51
*** fawadkhaliq has quit IRC10:52
openstackgerritAndrea Frittoli proposed openstack-infra/subunit2sql: Add pagination support for the runs query  https://review.openstack.org/22434410:53
*** dtantsur is now known as dtantsur|lunch10:55
*** fawadkhaliq has joined #openstack-infra10:55
*** yfried is now known as yfried|afk11:00
*** MarkAtwood has joined #openstack-infra11:06
*** romcheg has quit IRC11:07
*** ozamiatin has quit IRC11:07
*** dguitarbite has quit IRC11:08
*** dguitarbite has joined #openstack-infra11:10
tonybttx: Wow!11:10
*** yfried|afk is now known as yfried11:11
* tonyb is very surprised that was enough to do it.11:11
*** IvanKliuk has left #openstack-infra11:11
r1chardj0n3s\o11:12
r1chardj0n3syay11:13
*** MarkAtwood has quit IRC11:13
*** MarkAtwo_ has joined #openstack-infra11:13
*** doug-fish has quit IRC11:14
*** MarkAtwood has joined #openstack-infra11:14
*** doug-fish has joined #openstack-infra11:14
*** matbu_ has joined #openstack-infra11:17
openstackgerritAndrey Pavlov proposed openstack-infra/project-config: ec2api job: enable voting of py34 for master branch  https://review.openstack.org/22444111:17
*** MarkAtwo_ has quit IRC11:17
*** nadya has quit IRC11:19
*** matbu has quit IRC11:20
*** nadya has joined #openstack-infra11:20
*** glauco_ has joined #openstack-infra11:22
*** e0ne has quit IRC11:23
*** oomichi has quit IRC11:24
openstackgerritAntoine Musso proposed openstack/diskimage-builder: Prettyfy source-repositories doc  https://review.openstack.org/22457811:25
*** hashar has quit IRC11:25
*** markvoelker has joined #openstack-infra11:26
*** glauco_ has quit IRC11:27
*** dprince has joined #openstack-infra11:28
*** claudiub has joined #openstack-infra11:30
*** markvoelker has quit IRC11:31
*** Zhongjun has joined #openstack-infra11:31
*** bexelbie_ is now known as bexelbie11:32
*** zeih has joined #openstack-infra11:33
*** weshay_xchat has quit IRC11:35
*** dzamboni has joined #openstack-infra11:35
*** ldnunes has joined #openstack-infra11:35
*** weshay_xchat has joined #openstack-infra11:37
*** zeih has quit IRC11:38
*** EricGonczer_ has joined #openstack-infra11:40
*** jordanP has quit IRC11:40
*** sdake_ has joined #openstack-infra11:43
*** geoffarnoldX is now known as geoffarnold11:46
*** markvoelker has joined #openstack-infra11:47
*** sdake has quit IRC11:47
*** doug-fish has quit IRC11:48
ttxjeblair, fungi: I don't have SSH access to 23.253.49.144 (the odsreg machine) yet -- if that is setup I can probably push the last manual steps in11:49
*** thorst has joined #openstack-infra11:50
*** amuller has joined #openstack-infra11:50
*** nadya has quit IRC11:50
*** EricGonczer_ has quit IRC11:50
*** EricGonc_ has joined #openstack-infra11:50
fungittx: i can take a look in an hour or so. trying to get the voter rolls generated and need to drop my wife off at her office, but can dig into it once i get back11:51
*** gordc has joined #openstack-infra11:52
*** romcheg has joined #openstack-infra11:53
ttxhmm, maybe that was a test host, not the real thing11:54
ttxmaybe better to wait for jeblair11:54
*** trown|outttypeww is now known as trown11:55
openstackgerritMerged openstack/requirements: Update our handling of pbr.  https://review.openstack.org/22433011:59
*** fawadkhaliq has quit IRC12:01
*** baoli has joined #openstack-infra12:04
*** e0ne has joined #openstack-infra12:04
*** fhubik is now known as fhubik_brb12:05
openstackgerritBob Ball proposed openstack-infra/project-config: Add fuel-plugin-xenserver project  https://review.openstack.org/22415112:06
*** bswartz has quit IRC12:10
openstackgerritOpenStack Proposal Bot proposed openstack-dev/hacking: Updated from global requirements  https://review.openstack.org/22459512:10
openstackgerritOpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements  https://review.openstack.org/22460612:11
*** amotoki has quit IRC12:12
*** sdake_ is now known as sdake12:13
*** ajmiller has quit IRC12:15
*** fawadkhaliq has joined #openstack-infra12:15
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from global requirements  https://review.openstack.org/22468512:17
mordredShrews: shade unittest are all broken because of te occ/ksa release - but it's because we were doing a very weird thing in the shade tests that isn't actually a valid thing to do12:19
mordredShrews: I'm working on a fix12:19
mordredShrews: (basically, we were instantiating a cloud with absolutely no config information at all)12:20
*** yamamoto has quit IRC12:20
mordredwhich worked for the unit tests - but isn't actually a thing12:20
*** glauco has joined #openstack-infra12:23
*** ociuhandu has joined #openstack-infra12:23
*** romcheg has quit IRC12:23
claudiubfungi: hello. the os-win project repo merged yesterday: https://review.openstack.org/#/c/209204/  can you add me to its core and release groups? Thanks!12:23
*** burgerk has joined #openstack-infra12:25
*** glauco has quit IRC12:28
openstackgerritMonty Taylor proposed openstack-infra/shade: Start using keystoneauth for keystone sessions  https://review.openstack.org/18317612:28
mordredShrews: ^^ that should do it12:28
*** jsavak has joined #openstack-infra12:30
*** fhubik_brb is now known as fhubik12:31
*** markvoelker has quit IRC12:31
*** romcheg has joined #openstack-infra12:31
*** ociuhandu has quit IRC12:31
*** weshay_xchat is now known as weshay12:32
*** doug-fish has joined #openstack-infra12:34
*** LimorStotland has joined #openstack-infra12:34
LimorStotlandHi all... i have a question ... is there any way to rerun gate test?12:35
*** aysyd has joined #openstack-infra12:35
*** achanda has joined #openstack-infra12:35
openstackgerritMonty Taylor proposed openstack-infra/shade: Use the get_auth function from occ  https://review.openstack.org/22112612:36
*** cburroughs has joined #openstack-infra12:38
*** hashar has joined #openstack-infra12:38
*** dizquierdo has quit IRC12:39
*** EricGonc_ has quit IRC12:42
*** nadya has joined #openstack-infra12:43
*** geoffarnold is now known as geoffarnoldX12:46
*** dprince has quit IRC12:47
*** hamzy has quit IRC12:47
*** mdbooth has quit IRC12:48
*** dtantsur|lunch is now known as dtantsur12:48
openstackgerritMonty Taylor proposed openstack-infra/shade: Trap exceptions in helper modules  https://review.openstack.org/22471312:48
EmilienMgood morning infra, I have an outstanding request to unblock our stable/x branches CI: https://review.openstack.org/#/c/224397/12:49
EmilienMmordred: I was wondering if you also tested cloudwatt (french compute & storage public cloud)12:49
*** mdbooth has joined #openstack-infra12:50
*** amotoki has joined #openstack-infra12:50
*** julim has joined #openstack-infra12:50
*** edmondsw has joined #openstack-infra12:51
Shrewsmordred: hrm12:51
mordredEmilienM: I have not yet- it's now on my list12:51
EmilienMmordred: they use OpenContrail for SDN & Neutron plugin12:52
EmilienMmight be interesting to benchmark how they allow to use public network12:52
fungiLimorStotland: leave a review comment starting with the word "recheck"12:53
*** gordc has quit IRC12:53
LimorStotlandThanks12:53
EmilienMmordred: i've been working with us when I was eNovance - let me know if you need to reach out someone from there12:53
mordredEmilienM:12:54
mordredBienvenue et merci de votre inscription sur Cloudwatt !12:54
*** Daisy has joined #openstack-infra12:55
*** redixin has quit IRC12:55
mordredEmilienM: boo. it does not take Amex :)12:56
EmilienMmordred: Bienvenue en France12:56
EmilienMmordred: ;-)12:56
openstackgerritBob Ball proposed openstack-infra/project-config: Add fuel-plugin-xenserver project  https://review.openstack.org/22415112:56
*** kgiusti has joined #openstack-infra12:57
*** pradk has joined #openstack-infra12:58
openstackgerritMerged openstack-infra/project-config: zuul/layout: puppet openstack updates  https://review.openstack.org/22439712:58
*** ociuhandu has joined #openstack-infra12:58
fungiclaudiub: i've added you to those groups now12:58
*** jolsen has joined #openstack-infra12:59
*** glauco has joined #openstack-infra13:02
*** yamamoto has joined #openstack-infra13:02
mordredEmilienM: aww. cloudwatt reuqires floating ips.13:03
*** xyang1 has joined #openstack-infra13:03
* mordred shakes fist at EmilienM13:03
EmilienMmordred: afik, they use provider networks but I used to work on it long time ago - it might have changed13:03
*** gordc has joined #openstack-infra13:03
EmilienMiirc, when you spawned a VM, it was directly on the internet13:03
mordredEmilienM: nope. it's changed13:03
EmilienMdamn13:03
mordrednow if you go to create a vm, it says "you don't have any networks"13:04
*** rlandy has joined #openstack-infra13:04
EmilienMmordred: I'm sure they are really open to discussion, I can give you contacts there13:04
*** geoffarnoldX is now known as geoffarnold13:05
*** ccarrara has joined #openstack-infra13:05
*** geoffarnold is now known as geoffarnoldX13:05
*** rossella_s has quit IRC13:06
*** rossella_s has joined #openstack-infra13:06
openstackgerritBob Ball proposed openstack-infra/project-config: Add fuel-plugin-xenserver project  https://review.openstack.org/22415113:06
EmilienMfor the record, I sent to mordred in /query the email of their network architect, also neutron contributor13:07
*** ozamiatin has joined #openstack-infra13:09
odyssey4meodd, https://git.openstack.org/cgit/openstack/governance/commit/?id=b8b01adf2171596a03cd5fe3948867da071df834 is in, but it's not reflected yet at http://governance.openstack.org/reference/projects/openstackansible.html13:10
*** ddieterly has quit IRC13:11
*** ddieterly has joined #openstack-infra13:11
*** hdd has joined #openstack-infra13:15
*** fawadkhaliq has quit IRC13:16
*** zns_ has joined #openstack-infra13:17
openstackgerritBob Ball proposed openstack-infra/project-config: Add fuel-plugin-xenserver project  https://review.openstack.org/22415113:17
*** teran_ has joined #openstack-infra13:17
openstackgerritMerged openstack-infra/project-config: Added new Openstack repository for Fuel Zabbix Agents integration  https://review.openstack.org/22125913:17
openstackgerritMerged openstack/diskimage-builder: Add centos7 support for DIB_DISTRIBUTION_MIRROR  https://review.openstack.org/22399713:17
*** HeOS_ has quit IRC13:17
*** [HeOS] has joined #openstack-infra13:17
*** mattymo has quit IRC13:18
*** teran has quit IRC13:19
*** zhenguo has joined #openstack-infra13:20
*** ddieterly has quit IRC13:21
*** romcheg has quit IRC13:22
*** bltavares has joined #openstack-infra13:22
*** asselin_ has joined #openstack-infra13:22
*** mattymo has joined #openstack-infra13:23
*** hrou has joined #openstack-infra13:23
*** arxcruz has quit IRC13:24
*** romcheg has joined #openstack-infra13:24
*** eharney has quit IRC13:24
*** ajmiller has joined #openstack-infra13:24
*** fawadkhaliq has joined #openstack-infra13:25
*** romcheg has quit IRC13:26
*** glauco has quit IRC13:26
*** romcheg has joined #openstack-infra13:28
*** cdent has quit IRC13:28
*** dramalho has joined #openstack-infra13:28
openstackgerritMerged openstack-infra/tripleo-ci: Use a specific mirror for CentOS builds  https://review.openstack.org/22400013:30
*** redixin has joined #openstack-infra13:30
*** dramalho has quit IRC13:30
*** alivigni has joined #openstack-infra13:31
*** openstackgerrit has quit IRC13:31
*** tongli has joined #openstack-infra13:31
*** andrearosa has joined #openstack-infra13:31
*** openstackgerrit has joined #openstack-infra13:31
*** ccarrara has quit IRC13:32
*** geoffarnoldX is now known as geoffarnold13:32
fungitristanC: tonyb: i'll go ahead and put together the final electoral rolls for: cinder, glance, ironic, keystone, mistral, neutron, oslo and also the general atc roll for the upcoming tc election13:32
*** mriedem has joined #openstack-infra13:32
tonybfungi: Thanks!13:32
tristanCfungi: many thanks :-)13:33
*** tiswanso has joined #openstack-infra13:33
*** zeih has joined #openstack-infra13:34
fungitonyb: tristanC: i'm basing it on the state of the governance repo at the sept-2015-elections tag ttx created a week ago13:34
*** dramalho has joined #openstack-infra13:35
*** sabeen has joined #openstack-infra13:35
tonybfungi: I believe that is the correct thing to do.13:35
*** julim has quit IRC13:37
claudiubfungi: thanks for adding me to the groups. :)13:38
*** ajmiller_ has joined #openstack-infra13:39
*** zeih has quit IRC13:39
*** achanda has quit IRC13:40
fungitonyb: tristanC: one glitch here... wiki says "over the Kilo-Liberty timeframe (September 18, 2014 06:00 UTC to September 18, 2015 05:59 UTC)"13:40
*** ajmiller has quit IRC13:40
*** ajmiller_ has quit IRC13:41
fungiSeptember 18, 2015 05:59 UTC is still ~16 hours from now13:41
tristanCfungi: yes, we pick that timeframe arbritarly to make it match with election dates13:41
fungiso i guess i should hold off generating the electoral rolls until then13:41
*** ajmiller has joined #openstack-infra13:42
*** burgerk has quit IRC13:44
*** dprince has joined #openstack-infra13:46
*** sigmavirus24_awa is now known as sigmavirus2413:46
tonybfungi: That makes sense to me but I'll let tristanC make the call.13:46
tristanCouch, would have been better to pick a day earlier. but since we communicate those dates, we'll have to stick with them13:46
tristanCtonyb: fungi: yes that will have to wait tomorrow then13:47
*** ajmiller has quit IRC13:47
*** hdd has quit IRC13:47
mriedemjhesketh: can you hit this again now that pep8 is happy? https://review.openstack.org/#/c/224162/13:48
*** jpeeler has joined #openstack-infra13:49
*** jpeeler has quit IRC13:49
*** jpeeler has joined #openstack-infra13:49
*** stevemar has joined #openstack-infra13:50
mordredleast favorite thing - waiting on a patch to land after all the tests have finished13:51
mriedemleast favorite, of all things?13:51
openstackgerritMerged openstack-infra/project-config: Don't run python jobs for old Fuel Client  https://review.openstack.org/22398013:52
openstackgerritMerged openstack-infra/tripleo-ci: Remove code to pull clone repositories  https://review.openstack.org/22240613:54
openstackgerritBob Ball proposed openstack-infra/project-config: Add fuel-plugin-xenserver project  https://review.openstack.org/22415113:55
*** hichihara has joined #openstack-infra13:56
*** mtanino has joined #openstack-infra13:58
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: Add devstack-plugin-bdd to project config  https://review.openstack.org/22473813:58
*** jyuso1 has quit IRC14:00
*** e0ne has quit IRC14:01
*** eharney has joined #openstack-infra14:05
*** julim has joined #openstack-infra14:05
*** dramalho has quit IRC14:05
*** peristeri has quit IRC14:06
*** e0ne has joined #openstack-infra14:06
mordredSergeyLukjanov, ttx: it's really painful waiting for the US to wake up - how do you do it on a regular basis?14:07
* Clint snickers.14:08
openstackgerritMateusz Matuszkowiak proposed openstack-infra/project-config: change verify group to fuel-plugins-ci  https://review.openstack.org/22413614:09
openstackgerritDmitry Tantsur proposed openstack/requirements: Bump upper-constraint for python-ironic-inspector-client to 1.2.0  https://review.openstack.org/22314314:09
*** gordc has quit IRC14:10
*** Daisy has quit IRC14:11
*** gordc has joined #openstack-infra14:11
*** burgerk has joined #openstack-infra14:11
EmilienMi'm not sure but is there a DNS issue? when I go on https://trystack.org/ I land in the OpenStack Security page14:11
*** glauco has joined #openstack-infra14:12
Clintsounds more like a vhost issue14:12
pabelangeryes14:12
pabelangerlooks like it14:12
EmilienMdamn14:12
pabelangerwhen we did trystack.o.o we didn't do https14:12
pabelangereasy fix14:12
pabelangerbut we need a cert14:12
ttxmordred: I know, right14:14
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: Add devstack-plugin-bdd to project config  https://review.openstack.org/22473814:14
*** Piet has quit IRC14:14
*** dramalho has joined #openstack-infra14:15
*** regXboi has joined #openstack-infra14:15
openstackgerritPaul Carver proposed openstack-infra/irc-meetings: Add more details to the Service Function Chaining meeting  https://review.openstack.org/22474714:16
openstackgerritBob Ball proposed openstack-infra/project-config: Add fuel-plugin-xenserver project  https://review.openstack.org/22415114:17
*** burgerk_ has joined #openstack-infra14:17
*** IvanKliuk has joined #openstack-infra14:17
fungiEmilienM: there is no https://trystack.org/ only http://trystack.org/14:17
fungipabelanger: ^14:18
pabelangerEmilienM: Clint: it should be noted, that all vhosts on static.o.o if you use https will redirect to security.o.o14:18
pabelangerso, it is not limited to trystack14:18
fungialso we'd need to do sni since we already have an https site on that apache server and it has only one global ip address14:18
EmilienMfungi: yeah but when I used it in the past it was https14:18
fungiEmilienM: interesting, that was probably not called out as a requirement when we redirected it14:19
Clintsni's pretty widespread now14:19
BobBallfungi: And I thought you were going to say there is no trystack there is only dostack...14:19
IvanKliukColleagues, I got problem with installing 'hacking' on CI http://logs.openstack.org/31/205131/5/check/gate-python-fuelclient-pep8/ee4612e/console.html14:19
IvanKliukERROR: could not install deps [hacking==0.7]14:19
*** ccarrara has joined #openstack-infra14:19
*** MarkAtwood has quit IRC14:19
*** mmedvede has joined #openstack-infra14:19
IvanKliukCan you please help?14:19
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: Add devstack-plugin-bdd to project config  https://review.openstack.org/22473814:20
pabelangerIvanKliuk: looks like a networking issue14:20
pabelangerNo route to host14:20
*** burgerk has quit IRC14:20
fungiClint: yep, it's mostly just a matter of replacing our existing security.o.o cert with a new one that the ca allows additional subjectaltnames14:20
fungiwhich is more expensive, but not terribly so. i've already checked into it with the ca we're using14:20
pabelangerfungi: Ya, that would likely be the simplest way14:20
fungiBobBall: oops! i totally missed the opportunity there14:21
Clintfungi: if only letsencrypt would mature faster14:21
BobBall:P14:21
*** Daisy has joined #openstack-infra14:21
IvanKliukpabelanger: how can I be resolved?14:21
IvanKliukit*14:21
*** achanda has joined #openstack-infra14:22
fungiClint: they issued their first cert this week14:22
fungii think14:22
fungifirst non-experimental/production cert anyway14:23
Clintyeah, not cross-signed though14:23
*** amotoki has quit IRC14:23
fungiif only cacert.org could have thrived instead of merely highlighting how the browser and ca cabal is not in any way an open community14:23
*** dkranz has joined #openstack-infra14:24
clarkbdont worry firefox will just negotiate ad hoc tls for us and never have any bugs14:24
Clinti dunno, you needed a blackbelt in bureaucracy to participate in that thing14:24
pabelangerIvanKliuk: not sure. looks like http://pypi.region-b.geo-1.openstack.org is down?14:24
Clintcacert, not bug-free firefox14:25
clarkbits actually a neat feature unfortunate that that bug occured in the first release of it14:25
pabelangerfungi: clarkb: any current issues with pypi mirrors?14:25
fungipabelanger: you tell us14:25
*** achanda has quit IRC14:25
pabelangerfungi: I see what you did there14:25
fungipabelanger: IvanKliuk: yeah, sure looks unresponsive to me14:26
pabelangerfungi: Ya, cacti doesn't look happy about it either14:26
fungithat's not good14:26
fungican't ssh into it14:26
*** markvoelker has joined #openstack-infra14:26
fungii wonder whether it's just that machine or if hpcloud is suffering somehow14:26
clarkblast time that happened hp deleted out account14:27
fungidoesn't even respond to ping14:27
fungile sigh14:27
clarkbor suspended it14:27
*** jordanP has joined #openstack-infra14:27
pabelangerclarkb: wow14:27
clarkbjeblair did you forget to update the email addr :)14:27
*** tjcocozz has joined #openstack-infra14:27
clarkbfungi I am on phone but I would check if account works14:28
*** hdd has joined #openstack-infra14:28
fungiERROR (Unauthorized): Account is inactive (HTTP 401)14:28
clarkbya...14:29
fungiyep14:29
clarkbwendar ^14:29
Clintruh-roh14:29
fungithat's what i get when trying to authenticate as openstackci2 for tenant openstackci2-project114:29
*** dramalho has quit IRC14:29
openstackgerritTony Breeds proposed openstack/requirements: Add a warning near oslo.utils.  https://review.openstack.org/22475714:29
clarkbwe should maybe redeploy mirror in the slave account at least that way they die together14:29
*** esker has joined #openstack-infra14:30
*** mtanino has quit IRC14:31
openstackgerritTony Breeds proposed openstack/requirements: Add a warning near oslo.utils.  https://review.openstack.org/22475814:31
*** mmedvede has quit IRC14:31
*** bltavares has quit IRC14:32
clarkbfungi should we disable hpcloud in nodepool?14:32
fungii am doing that now14:32
*** geoffarnold is now known as geoffarnoldX14:32
clarkbok let me actually wake up and find food so I can be useful in abit14:33
*** ccarrara has quit IRC14:33
*** ildikov has quit IRC14:33
*** glauco has quit IRC14:34
*** rfolco has joined #openstack-infra14:34
fungiset max-servers to 0 for all 5 hpcloud networks and have deleted all ready nodes there14:35
clarkbcool14:35
fungipuppet agent is disabled14:35
fungichange on teh way to do the same14:35
*** btully has joined #openstack-infra14:35
openstackgerritZara proposed openstack-infra/storyboard-webclient: Subscriptions WIP  https://review.openstack.org/22356214:35
*** ajmiller has joined #openstack-infra14:36
*** dizquierdo has joined #openstack-infra14:36
clarkbthis is a benefit to the ovh model of one account multiple users14:37
fungitoo bad ovh gra1 isn't already completely in place and tested for us14:37
mordredfungi, clarkb is that ... sigh14:39
mordredlet me ping some humans14:39
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Temporarily disable hpcloud in nodepool  https://review.openstack.org/22476614:39
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Revert "Temporarily disable hpcloud in nodepool"  https://review.openstack.org/22476714:39
purpHi, mordred. Consider me pinged.14:39
*** tonytan4ever has joined #openstack-infra14:39
mordredpurp: yay!14:39
*** ashleigh has joined #openstack-infra14:39
* mordred does the happy purp dance14:39
*** lawrancejing has quit IRC14:39
anteayapurp: thanks14:40
purpI need the account id/username via email so I can get this to someone to look at.14:40
mordredpurp: on it14:40
mordredclarkb: do we know account id?14:40
openstackgerritMichael Xin proposed openstack-infra/project-config: Add a new project syntribos  https://review.openstack.org/22035114:41
fungimordred: openstackci2-project1 is the "tenant name"14:41
*** cloudtrainme has joined #openstack-infra14:41
pabelangerfungi: just, just noticed the work jeblair did with OVH. Exciting14:41
clarkband openstackci2 the account name14:41
fungiyeah, not sure which is the one they need14:41
*** Daisy has quit IRC14:42
purpThanks, clarkb. Relaying.14:42
*** rfolco has quit IRC14:42
fungiis there a many:many relationship model for api accounts and tenants in keystone? some providers seem to do many:one and some seem to do one:many14:43
mordredclarkb, fungi: what about openstackjenkins2 ?14:43
clarkbmordred seemed fine14:43
fungimordred: seems to still be working because the nodes are being booted there and running jobs14:43
openstackgerritPino Toscano proposed openstack-infra/tripleo-ci: [WIP] Generate overcloud image outside the undercloud  https://review.openstack.org/22416614:43
mordredfungi: yes14:43
clarkbI think that account is actually aet up properly14:43
mordredfungi: awesome. so it's just the mirror14:43
ihrachysinfra pypi cache broken? http://logs.openstack.org/09/224209/2/check/gate-neutron-pep8/f7e112c/console.html14:43
clarkbbecause it didnt break last time either14:43
fungiit's just the tenant for our long-lived servers (notably our pypi mirror) that is currently broken14:43
mordredihrachys: yes. we're having some issues with our mirror in hpcloud and are working on it14:43
ihrachyscool, thanks14:44
*** yfried is now known as yfried|afk14:44
*** ashleigh has quit IRC14:44
fungiihrachys: we've stopped running jobs in hpcloud now until it's resolved, so the failures should subside14:44
ihrachysso rechecks justified?14:44
mordredihrachys: yup14:45
*** Daisy has joined #openstack-infra14:45
*** dangers_away is now known as dangers14:45
*** Daisy has quit IRC14:45
openstackgerritThierry Carrez proposed openstack-infra/puppet-statusbot: Enable #success feature from statusbot  https://review.openstack.org/22476814:45
openstackgerritThierry Carrez proposed openstack-infra/system-config: Enable the #success feature in statusbot  https://review.openstack.org/22476914:45
jeblairrestarted carbon-cache14:46
mordredfungi: so - users and projects can be many to many - a project is a place in which you put resources, such as vms. a user is an acocunt that can have a role in a project14:46
clarkbwe can also update dns temporarily to point at iad14:46
ttxclarkb: ^ those should be enough to enable the #success feature -- we might want to land them in a particular order though I could not decide which.14:46
fungiclarkb: not a terrible idea14:46
jheskethMorning14:46
clarkbI think iad is closest as the crow flies14:47
ttxAlso wondering if we enforce '=>' alignment and whether I should correct untouched lines.14:47
*** skolekonov is now known as _skolekonov14:47
jeblairttx: not on system-config14:47
*** geoffarnoldX is now known as geoffarnold14:47
clarkbttx we do in everything but system-config14:47
jheskethmriedem: looking14:47
*** mmedvede has joined #openstack-infra14:47
ttxok, so that should work14:47
fungiclarkb: i'm tempted to wait a bit and see if hpcloud comes back with a quick resolution before making dns changes we need to undo again14:47
jeblairttx: and yeah, .144 is a test host.  i can spin up a real one today.  what should it be called?14:47
*** ildikov has joined #openstack-infra14:48
ttxjeblair: your guess is as good as mine14:48
fungijeblair: ttx: any reason the module shouldn't be deployed to the summit prep server? or should it not coexist with cheddar?14:48
clarkbfungi +114:48
*** btully has quit IRC14:49
ttxfungi: current cheddar host is manually deployed, so not sure how that would play out. But we can certainly try14:49
fungi(noting that the cheddar deployment on the summit prep server needs puppeting still, though the server itself is in puppet)14:49
ttxyeah,n the cheddar stuff is not even on Gerrit14:49
*** Daisy has joined #openstack-infra14:50
fungioh, ugh14:50
fungimaybe we just avoid touching that then14:50
ttxthose 6 months sure did fly14:50
fungifor now14:50
*** oomichi has joined #openstack-infra14:51
*** Piet has joined #openstack-infra14:51
*** achanda has joined #openstack-infra14:52
jeblairodsreg.o.o then?14:53
jeblairfor now14:53
pabelangerfungi: is there a reason not to add the other mirrors to PIP_USE_MIRRORS for jenkins slaves? In an effort to find an available mirror the if the default goes down?14:53
ttxjeblair: sure14:53
pabelangerI guess we only use the local pypi only14:53
*** dims_ has quit IRC14:53
jrollmordred: out of curiousity, what's the status with phabricator stuff, anything I can do to help move that?14:54
clarkbpabelanger the way pip works evwry single one is contacted14:54
clarkbmirrors arent for ha in your pypi config14:54
*** bhunter71 has joined #openstack-infra14:54
fungipabelanger: well, one reason is so that situations like this don't fly under the radar until something is broken with several of our mirror servers14:54
clarkbfungi the joba would fail abyways14:55
fungiyeah, that too14:55
clarkbbecause pip would ask for the index fail and exit14:55
pabelangerfungi: Ya, thats the downside I see. Sometime, you want things to break and scream so you need to fix them14:55
*** dims_ has joined #openstack-infra14:55
*** lucasagomes has joined #openstack-infra14:56
*** btully has joined #openstack-infra14:56
*** rfolco has joined #openstack-infra14:56
*** armax has joined #openstack-infra14:57
*** asselin__ has joined #openstack-infra14:57
*** rick-chen has joined #openstack-infra14:58
jeblairClint: ^ see question from jroll14:58
*** asselin_ has quit IRC14:58
*** vgridnev has quit IRC14:59
clarkbI think tge idea is that you really have non overlappying indexes and need to look at all of them to find the correct packages14:59
clarkblike if you have a bunch of non open source python hosting your web app14:59
*** unicell1 has joined #openstack-infra15:00
*** dramalho has joined #openstack-infra15:00
clarkb* for hosting15:00
*** unicell has quit IRC15:00
*** tiswanso has quit IRC15:00
fungiright, or "internal" pip repositories in your company for locally-produced packages15:00
rick-chenanteaya: HI, Do you know how to revert the CI verify machine to be clean for next testing?15:00
*** glauco has joined #openstack-infra15:01
fungirick-chen: we delete the machine and boot a fresh one15:01
fungifor each job run15:01
*** tiswanso has joined #openstack-infra15:01
*** ccarrara has joined #openstack-infra15:01
*** yfried|afk is now known as yfried15:02
*** Swami has joined #openstack-infra15:02
fungirick-chen: because especially if you're running a job which has root access to the machine and is testing untrusted code change proposals, the job could render the worker completely unusable/unreachable (we see that happen pretty often in fact)15:02
rick-chenfungi: Use nodepool?15:02
fungirick-chen: yes, that's why we wrote nodepool15:02
*** zz_dimtruck is now known as dimtruck15:03
rick-chenfungi: Do you have detail how to build up node pool document?15:04
*** bltavares has joined #openstack-infra15:04
*** jsavak has quit IRC15:04
fungirick-chen: nodepool has documentation at http://docs.openstack.org/infra/nodepool/15:04
pabelangerwouldn't it be cool is nodepool also built the targets too. Oh, you need a new jenkins master, let me launch that for you15:05
*** jsavak has joined #openstack-infra15:05
Clintjeblair: thanks15:05
hasharrick-chen: the doc is part of the source repo openstack-infra/nodepool.git   you can build it locally using:  tox -e docs15:05
rick-chenfungi: I use vphere plugin for esxi, but I got the problem that cannot post the result to the gerrit review windows.15:05
anteayarick-chen: I don't see you in the #openstack-cinder channel15:05
fungipabelanger: yep, we talked about that, though for jenkins the trick is that it's nontrivial to automate a new deployment of one15:05
hasharrick-chen: resulting HTML will end up in /doc/build/html/15:05
anteayarick-chen: these questions are great questions for the other cinder third party ci operators15:06
Clintjroll: http://docs-draft.openstack.org/72/219372/7/check/gate-infra-specs-docs/5e9573f//doc/build/html/status/maniphest.html is slightly out of date15:06
smcginnisrick-chen: Just joining. What's the question?15:06
anteayaand asking them in #openstack-cinder is a great way to get to know cinder developers15:06
rick-chenanteaya: OK, Got.15:06
anteayasmcginnis: thank you15:06
clarkband jenkins is not required15:06
pabelangerfungi: Ya, I'm going to try some stuff with ansible to see if I can automate some of the manual steps. But, I might try adding some code into nodepool too15:06
smcginnisrick-chen: If it's cinder specific, let's head over to -cinder.15:06
*** zhenguo has quit IRC15:06
anteayasmcginnis: rick-chen is asking about how to get a clean environment for testing15:06
hasharanteaya: is there any third parties using Nodepool already?15:07
anteayasmcginnis: we directed him to look at nodepool15:07
anteayahashar: many15:07
smcginnisanteaya: OK, cool. Thanks!15:07
anteayasmcginnis: following up in cinder is a great way to integrated with the cinder folks15:07
anteayain my opinion15:07
hasharanteaya: damn I thought I was one of the early adopters :-}   Good to see it matches the needs of other CIs15:07
anteayahashar: you were15:07
anteayaand still are an early adopter of many of our tools15:08
anteayabut eventually other folks climb onto a good thing :)15:08
hasharI turned it on for prod usage yesterday \O/15:08
fungipabelanger: at the moment we've not really seen a lot of benefit to continuing to do anything more than limp our use of jenkins along until we finish getting rid of it. certainly scaling jenkins masters in and out is an infrequent enough task so as to be beneath the threshold of necessity15:08
anteayahashar: yay!15:08
jrollClint: cool, ty, I'm excited for this15:09
jheskethto save me reading scrollback... did we just do a nodepool restart?15:09
*** achanda has quit IRC15:09
Clintjroll: also feel free to code-review https://review.openstack.org/#/c/219372/ if you have opinions about that page15:10
pabelangerfungi: Ya, would be more a task to see if it could be done vs pushing it back upstream.15:10
fungijhesketh: nodepoold hasn't been restarted for a week, no15:10
jrollthere are currently pypi mirror issues yes?15:10
fungijhesketh: i reloaded it to pick up the hpcloud max-servers=0 config change but that's it15:10
jheskethfungi: what's up with the broken job stats on http://status.openstack.org/zuul/15:11
hasharand now for my 2 million dollars morning question: could Zuul rely on OpenStack Mistral for workflow / job definitions?15:11
*** dkranz has quit IRC15:11
mordredhashar: out current plan is to rely on ansible15:12
mordredhashar: our15:12
mordredhashar: does mistral support operating against clouds it's not installed in?15:12
fungijhesketh: looks like carbon-cache had a sad on graphite.o.o and jeblair logged in and restarted it15:12
jheskethfungi: okay, good to know, thanks :-)15:12
fungijhesketh: same oom condition we seem to hit there frequently15:12
*** mariojv has joined #openstack-infra15:13
anteayajhesketh: what are your plans for the day? I was soon going to drive downtown15:13
jheskethanteaya: was just going to do bits of work, pack and head off... I was contemplating wandering around though15:14
mordredjhesketh: with you in the US and me in Europe, our globla core distributoin is strange15:14
hasharmordred: I have no idea. Seems it can be run without an OpenStack environment.   That needs a worker on the clients though  where as I think Ansible just compile everything and send the resulting blob;  clienteles?15:14
jheskethanteaya: but I wouldn't be ready for a bit, so I might just do my own thing :-)15:15
mordredhashar: yah. we want to avoid agents because they pollute the test nodes15:15
anteayasounds good15:15
mordredhashar: this is one of the things we really like about the ansible job definition approach15:15
hasharmordred: this way you only need a ssh connection and python right?15:15
mordredhashar: allows the worker machines to be extremely minimal in what exists outside of the job definition15:15
mordredhashar: yah15:15
hashar\O/15:15
mordredhashar: and I think we're at the point where both of those are fairly standard :)15:16
hasharmordred: so will Zuul define the ansible tasks ?  Or would you still use JJB to generate them ?15:16
jheskethmordred: yes, but so long as we all keep moving around, we should be fine15:16
jheskethmordred: it's like musical chairs, we can't stop moving15:16
mordredhashar: we have a whole spec! one sec, I get you a link15:16
mordredhashar: http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html15:16
hasharmordred: yeah the Zuul v3 spec, I haven't found time to read it yet :-((((15:17
mordredhashar: short version - we will have zuul define things about the mapping of job content to workers15:17
mordredhashar: this will also describe how many of what type of machine you might want15:17
openstackgerritMerged openstack-infra/irc-meetings: Add more details to the Service Function Chaining meeting  https://review.openstack.org/22474715:17
mordredhashar: but the actual content itself will just be ansible roles/playbooks15:17
mordredhashar: so that it would be conceivable that you could use your real deployment code as part of your CI15:17
pabelangerhashar: I recommend the read! I am super excited for it15:18
*** DericHorn-HP has joined #openstack-infra15:18
pabelangermordred: !!!!15:18
hasharI will for sure15:18
hashargotta upgrade Wikimedia Zuul to catch up with six months of dev you guys did15:18
pabelangerusing real depolyment code to do both production and testing environments is super awesome15:18
*** tongli has quit IRC15:19
pabelangerthat's the main reason for me right now to use ansible15:19
*** alexpilotti has quit IRC15:19
*** angdraug has joined #openstack-infra15:20
hashar  queue: integrated  # Shared queues are manually built  Â \O/15:20
*** hemnafk is now known as hemna15:21
* mordred loves making hashar excited15:21
hasharmordred: you could even have Zuul to ship built-in jobs  such as the tox runner. Would save third parties a bunch of time I guess15:22
*** johnsom has joined #openstack-infra15:22
mordredyup15:22
*** MarkAtwood has joined #openstack-infra15:22
mordredttx, morgan, dhellmann: https://review.openstack.org/#/c/224739/ if you get a sec15:22
*** madong_ has quit IRC15:23
*** madong_ has joined #openstack-infra15:24
*** alexpilotti has joined #openstack-infra15:24
*** dkranz has joined #openstack-infra15:24
*** amitgandhinz has joined #openstack-infra15:24
morganmordred: yep was on my list for today but ++ provided Jenkins doesn't get in the way15:25
*** puranamr has joined #openstack-infra15:25
mordredmorgan: thanks!15:27
*** ozamiatin has quit IRC15:27
*** yfried has quit IRC15:27
fungimordred: see my comment to your 224743 governance change. this may have consequences you did not intend (or perhaps you did intend, i don't know)15:28
fungimordred: i'm not opposed to the cleanup, but the resulting retroactive invalidation should probably be mentioned in the commit message15:29
*** krtaylor has quit IRC15:29
mordredfungi: nod. I given the timing - I would be fine holding off on voting such a thing until after the current TC election - and I agree, it should definitely be mentioned15:30
*** hyakuhei has joined #openstack-infra15:30
fungimordred: well, luckily i'll be generating the electoral rolls for the upcoming tc election tomorrow, at the same time i do so for the ptl elections, and the review is unlikely to get approved before the next tc meeting anyway15:31
*** MarkAtwo_ has joined #openstack-infra15:31
*** earlephilhower has joined #openstack-infra15:32
* hyakuhei is apparently unable to track dates or effectively grok email :(15:32
*** sridhar_ram has joined #openstack-infra15:32
*** fhubik has quit IRC15:32
*** mdenny has joined #openstack-infra15:32
fungialso the final batch of complimentary contributor discount codes for the upcoming summit has already been sent, so no impact there until the "n" summit timeframe anyway15:32
*** cbader has joined #openstack-infra15:33
mordredhyakuhei: dates and email are both hard15:34
anteayawhich is why for critical dates and times I confirm with a few others to ensure I am thinking the right thing15:34
anteayaso confirmation is good15:34
hyakuheioh don’t even get me started on timeframes :(15:34
anteayaha ha ha15:34
hyakuheis/frames/zones/15:34
*** bpokorny has joined #openstack-infra15:35
*** zeih has joined #openstack-infra15:35
fungithis is why i try to do things at the earliest possible moment rather than leaving them for the last possible moment. that way if i miscalculate, at worst i look proactive ;)15:35
anteayagood strategy!15:36
*** mmedvede has quit IRC15:36
purpHi fungi jeblair et al. HP Cloud should be available to you again.15:37
fungihyakuhei: i assumed you were talking about einstein spacetime frames of reference15:37
mordredpurp: yay!15:37
anteayapurp: yay \o/15:37
anteayapurp: thank you15:37
* mordred hands purp a completely uneaten basket of fruit15:37
anteayawow15:37
hyakuheifungi: those too!15:37
anteayasomething useful15:37
purpHeh. Thanks. I did so little.15:37
anteayapurp: timing15:37
anteayait is all timing15:37
hyakuheiIn fact, if I’m travelling at lightspeed and you’re walking backwards I think you’ll find the election window is still open…15:37
anteayapurp: mordred never hands out useful things to anyone else15:38
anteayapurp: you're special15:38
Clinti don't think that's true15:38
Clintunless you think fuzzy llamas are not useful15:38
fungipurp: thanks! i'll check again now15:38
mordredanteaya: come on. you did not enjoy the somewhat unfrazzled aardvark?15:39
*** tongli has joined #openstack-infra15:39
*** zeih has quit IRC15:39
anteayait was handy15:39
anteayait eats ants15:40
anteayadoesn't it?15:40
*** tonytan4ever has quit IRC15:40
jeblairopenstackci2-hpcloud-region-b.sh  <-- this one, right?15:40
anteayawell llama fur is useful I think15:40
jeblairi still get ERROR (Unauthorized): Tenant not accessible (HTTP 401)15:40
fungijeblair: i get the same'15:40
*** devlaps has joined #openstack-infra15:40
fungipurp: is there some propagation delay?15:41
jeblairfungi, mordred: btw, i built gra1 on trusty without thinking... i think we have some trustification to do with our pypi mirror puppet15:41
anteayaokay I am running away for a bit15:42
anteayaI don't think anyone is waiting on me for anything15:42
anteayasee all y'all later15:42
*** dramalho has quit IRC15:43
fungijeblair: most likely issue will be accommodating apache 2.4isms in the vhost config15:43
*** glauco has quit IRC15:43
jeblairfungi: yep15:43
jeblairworking on that now15:43
fungii need to disappear to run some errands, but should be back within the hour15:43
*** Hal has quit IRC15:43
jeblairfungi, clarkb: looks like nodepool was not restarted?15:44
jeblairwhat's the status there, and when am i clear to do so? :)15:44
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: The apikey is now optional  https://review.openstack.org/22479615:44
jeblairmordred: ^15:44
mordredjeblair: you are clear to restart. the issues in the tests were related to fake config construction15:44
*** moravec has joined #openstack-infra15:44
mordredjeblair: or, rather, construction of configs for a scheme we do not use in production15:44
clarkbjeblair mordred fix didnt work. greghaynes fixed oscc instead which has a new release15:45
jeblairokay, so all i have to do now is wait a few hours for dib builds to complete, then we can see if ovh works15:45
clarkbbut did we merge pbr update?15:45
clarkbtgats what we were waiting on15:45
*** dramalho has joined #openstack-infra15:45
*** dramalho has quit IRC15:45
*** hamzy has joined #openstack-infra15:47
*** bltavares has quit IRC15:47
*** annegentle has joined #openstack-infra15:47
*** hodos|2 has joined #openstack-infra15:48
*** geoffarnold is now known as geoffarnoldX15:49
fungijeblair: clarkb: mordred: i've gone ahead and approved 224766 for now so that puppet agent can get enabled again15:49
*** IvanKliuk has left #openstack-infra15:49
fungii have a wip revert proposed which can be approved once we're sure the issue in hpcloud is resolved15:50
*** mmedvede has joined #openstack-infra15:50
*** Ala has quit IRC15:50
fungiat the moment pypi.region-b.geo-1.openstack.org is still not responding to ping, so not sure how long our account reenablement will take to be effective again15:50
*** geoffarnoldX is now known as geoffarnold15:50
*** puranamr has quit IRC15:51
*** oomichi has quit IRC15:51
purpfungi jeblair shouldn't be propagation delay. I'll check in tho15:51
fungianyway, really heading to do errands now15:51
clarkbmaybe we need to rebuild it?15:51
fungiclarkb: well, also still getting "ERROR (Unauthorized): Tenant not accessible (HTTP 401)" with that account so rebuilding it probably isn't possible yet15:51
clarkbah15:52
jeblairwow, erm.  nodepool throws an exception every time through the loop when there's a missing disk image...15:52
*** annegentle has quit IRC15:52
* jeblair switches to nodepool hacking15:53
*** annegentle has joined #openstack-infra15:53
greghaynesjeblair: oh?15:55
*** matrohon has joined #openstack-infra15:56
mriedemclarkb: fungi: i think we're going to need a force merge on stable/juno for a chicken/egg issue15:56
mriedemdetailed here https://bugs.launchpad.net/python-ceilometerclient/+bug/1494516/comments/815:56
openstackLaunchpad bug 1494516 in python-ceilometerclient "python-ceilometerclient 1.0.12 has uncapped oslo.utils which breaks unit test jobs on stable/juno" [Critical,New]15:56
greghaynesjeblair: it it when it tries to rm the missing image?15:56
*** krtaylor has joined #openstack-infra15:57
*** markus_z has quit IRC15:57
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use ubuntu-trusty as base for u-t-check in nodepool  https://review.openstack.org/22480315:58
jeblairclarkb, greghaynes: first there's that ^15:58
*** e0ne has quit IRC15:59
*** LimorStotland has quit IRC15:59
greghaynesok. I did notice yesterday that if an image is missing on disk itll try and rm it with no exception handling before it cleans up the db15:59
greghaynesso just loop on that15:59
clarkbjeblair your gra1 image is -check15:59
*** e0ne has joined #openstack-infra16:00
clarkbline 918ish16:00
*** [HeOS] is now known as HeOS16:00
*** ashleigh has joined #openstack-infra16:01
jeblairclarkb: hrm.  now i'm confused.16:01
*** serg_melikyan has joined #openstack-infra16:01
jeblairit should have uploaded an image yesterday, because there should already be an ubuntu-trusty image16:01
openstackgerritEmilien Macchi proposed openstack-infra/project-config: make gate-puppet-ironic-puppet-beaker-rspec-dsvm-trusty voting  https://review.openstack.org/22480516:02
*** purp has quit IRC16:02
*** sabeen has quit IRC16:02
clarkbjeblair not raw16:02
clarkbjeblair the new image type forces a rebuild16:03
greghaynesah, yep16:03
*** warpc__ is now known as warpc16:03
mriedemfungi: clarkb: ignore force-merge thing, i tihnk i have another way16:03
*** rlandy has quit IRC16:03
*** rick-chen has quit IRC16:04
jeblairokay.... so open questions are: why didn't that happen, and why are there 5 images showing up as building in the dib-image-list with only one running?16:04
*** yamamoto has quit IRC16:04
jeblairoh, are the other 4 queued?16:04
*** woodster_ has joined #openstack-infra16:04
clarkbyes they are serial16:04
jeblairthen there should be a new db state :)16:05
greghaynesI think we just query for any image of that type in state building16:05
*** sergmelikyan has quit IRC16:05
greghaynesyea, and if we find one we wait16:05
*** hichihara has quit IRC16:05
*** jlanoux has quit IRC16:05
greghaynestype is overloaded, any image with that name.16:05
clarkbit puts them in a Queue object and pulls one at atime16:05
jeblairclarkb: yeah, i knew that, i was just surprised the cli told me 5 were building16:05
clarkbwe can have popping off that be  a new state change16:05
jeblairit looks like you can infer that by the presence of a 'version' field16:06
*** dtantsur is now known as dtantsur|afk16:06
*** erikwilson has joined #openstack-infra16:06
*** moravec has quit IRC16:07
jeblairclarkb, greghaynes: i'm worride that image type differences are suppressed...16:08
*** moravec has joined #openstack-infra16:09
greghaynesjeblair: You sure its not related to the traceback?16:09
jeblairif you say that we need to do a new image build for a new type; it looks like the thing that goes into the queue is only a single record16:09
openstackgerritMerged openstack/requirements: Deal with underscores in names in edit-constraints  https://review.openstack.org/22425916:09
jeblairgreghaynes: no16:09
jeblairgreghaynes: not sure :)16:09
jeblairgreghaynes: well, tangentially --16:10
*** unicell1 has quit IRC16:10
jeblairgreghaynes: in that i think we're only checking the _name_ to see if an image is building and skipping in that case16:10
*** bexelbie has quit IRC16:10
*** hyakuhei has quit IRC16:11
jeblairgreghaynes: (we're 'skipping' incorrectly by throwing an exception, but even if we did what i think the code intended we would still only build ubuntu-trusty qcow and never get around to building u-t raw)16:11
greghaynesyea, I think that just serializes us though - in theory the main loop will pick up the missing image type I think16:11
greghayneshrm16:11
jeblairgreghaynes: line 2070 only queries by image name16:12
clarkbjeblair yes each image in the db has N formats on disk16:12
jeblairgreghaynes: oh, we're supposed to build all types at once i think16:12
clarkbyes16:13
jeblairline 86516:13
jeblairclarkb: ah, i misunderstood you16:13
openstackgerritAlexey Elagin proposed openstack-infra/project-config: change verify group to fuel-plugins-ci  https://review.openstack.org/22415916:13
*** Daisy has quit IRC16:13
jeblairclarkb: we _do_ need a new image built for adding a new type, but we don't need one build per-type16:13
*** purp has joined #openstack-infra16:13
openstackgerritMatt Riedemann proposed openstack/requirements: stable/juno: adjust python-ceilometerclient version range  https://review.openstack.org/22480716:13
clarkbcorrect16:13
mriedemtonyb: ^16:13
jeblairclarkb: next run we end up with one build producing multiple types16:13
clarkbyup16:13
*** unicell has joined #openstack-infra16:13
*** Daisy has joined #openstack-infra16:14
*** abregman_ has joined #openstack-infra16:14
greghaynesyea, we pass the multipe types in to dib at once16:14
jeblairokay, back to the drawing board for why this didn't happen yesterday :)16:14
*** hashar has quit IRC16:14
*** Daisy has quit IRC16:14
*** tonytan4ever has joined #openstack-infra16:15
*** abregman has quit IRC16:15
*** e0ne has quit IRC16:15
*** che-arne has quit IRC16:16
*** romcheg has quit IRC16:16
jeblairlooking at the logs... i think it's been sucessfully building ubuntu-trusty images all night16:16
jeblairit just consistently thinks they are missing16:16
openstackgerritKevin L. Mitchell proposed stackforge/python-jenkins: Allow setting the next build number  https://review.openstack.org/22427516:16
*** che-arne has joined #openstack-infra16:16
*** bharathm has joined #openstack-infra16:17
greghaynesjeblair: Is there a tb somewhere?16:17
*** rlandy has joined #openstack-infra16:18
openstackgerritDan Moravec proposed openstack-infra/subunit2sql: Add --verbose option to show sql statements generated  https://review.openstack.org/22431416:18
jeblairgreghaynes: nope (aside from the one about skipping)16:18
jeblair2015-09-17 07:02:56,227 WARNING nodepool.NodePool: Missing disk image ubuntu-trusty-check16:18
jeblairgreghaynes: i'm wondering if we've introduced a layer of abstraction in the config that doesn't actually exist -- maybe the provider image name needs to match the dib image name16:19
clarkbhrm I wonder if it has diskimage name, image name, and label name cobfused16:19
clarkbya that16:19
*** scheuran has quit IRC16:19
greghaynesjeblair: ok, so that is the method that seemed kind of broken to me16:19
greghayneswhen I was doing builder stuff yesterday16:19
*** sdake_ has joined #openstack-infra16:19
*** fitoduarte has joined #openstack-infra16:20
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Initial support for graph panel  https://review.openstack.org/22481116:20
greghaynesjeblair: What its going to do is check if the image exists on disk for all the types in the config (which it wont), then its going to try and rm the missing image and retry16:20
greghaynesand when I say retry its going to set found = false and return, so itll just wait for that method to be called again16:20
*** ifarkas has quit IRC16:20
*** jsavak has quit IRC16:20
*** moravec has quit IRC16:21
*** jsavak has joined #openstack-infra16:21
mriedemso is this an ok time to announce my candidacy for PTL of magnetodb?16:21
pabelangermordred: jeblair: do you mind re-approving https://review.openstack.org/#/c/223280/? queue gate failed last night, recheck is clean today16:22
greghaynesjeblair: actually, it does call buildImage after, so it should kick off a new build16:22
greghaynesjeblair: something I would check in the logs - does the filename dib makes match the filename in the row for the dib image it says is missing?16:23
*** unicell1 has joined #openstack-infra16:23
*** sdake has quit IRC16:23
greghaynesspecifically for the raw type, might be somethign wierd going on with that16:23
clarkbgreghaynes but we test this16:23
clarkbin dsvm and unittests16:23
greghaynesclarkb: for raw?16:23
clarkbtge specific type doesnt matter16:24
greghayneswell, if dib doesnt append a .raw for example16:24
clarkboh unless it does16:24
*** tiswanso has quit IRC16:24
clarkb:|16:24
clarkbdamnit dib16:24
* greghaynes is kind of just theorizing, might not be the case16:24
greghaynesI dont actually remember if dib does that16:24
*** Daisy has joined #openstack-infra16:24
*** tiswanso has joined #openstack-infra16:24
*** unicell has quit IRC16:25
jeblairi'm pretty sure it thinks it's missing because the image name (not the suffix) doesn't match16:25
*** tiswanso has quit IRC16:25
*** tiswanso has joined #openstack-infra16:25
clarkbour tests do qcow2 with real dib and qcow2 and vhd with the fake script dib in unittests16:26
jeblairyeah, we're hitting this: 2015-09-17 06:13:35,141 WARNING nodepool.NodePool: Missing disk image ubuntu-trusty-check16:26
jeblairon line 195916:26
jeblairfollowed by: 2015-09-17 06:13:35,143 DEBUG nodepool.NodePool: Queued image building task for ubuntu-trusty16:26
*** yamamoto has joined #openstack-infra16:26
greghaynesjeblair: that happens if it cant fine name + "." + type on disk16:26
purpfungi jeblair clarkb: Sorry, I'm too churned this morning for prolonged attention. Has the HP cloud account come back for you yet?16:26
greghaynesjeblair: line 194116:26
jeblairgreghaynes: i don't think it's making it past line 193216:27
greghayneslooks like dib should make a .raw though, so that theory is out16:27
greghaynesjeblair: oh, it just isnt finding any matching images at all? :(16:28
* dims_ checking in on zuul...16:28
*** pblaho has quit IRC16:28
*** romcheg has joined #openstack-infra16:28
jeblairpurp still getting ERROR (Unauthorized): Tenant not accessible (HTTP 401)16:28
purpThanks, jeblair. I'll go ask some further questions.16:29
greghayneswe could definitely put better debug info to say what specific image + type it thinks is missing16:29
clarkband ping still fails16:29
*** Daisy has quit IRC16:29
greghaynesjeblair: ok, so you dont see the image filename missing log message im guessing is why youre saying its not going around that loop at all16:30
jeblairgreghaynes: from existing log messages, i know that it thinks it's missing u-t-c, and so it decides to build u-t (which is correct).  so we handled the dinection in one place, not another.16:30
jeblairgreghaynes: correct, no 'missing filename' msg16:30
jeblairgreghaynes: the loop is "u-t-c" is missing; built "u-t"; "u-t" is being built; "u-t-c" is missing....16:31
*** fawadkhaliq has quit IRC16:31
jeblairgreghaynes: the loop is "u-t-c" is missing; build "u-t"; "u-t" is being built; "u-t" is built; "u-t-c" is missing....16:31
*** unicell1 has quit IRC16:32
mordredjeblair, clarkb: I just had a phone call but am bck - anything need my attention righ tnow?16:32
*** matrohon has quit IRC16:32
greghaynesjeblair: huh. Can you see the dib command it ends up running for u-t-c?16:32
jeblairgreghaynes: yes16:32
*** serg_melikyan has quit IRC16:32
greghaynesjeblair: and it has raw in the types?16:33
*** sergmelikyan has joined #openstack-infra16:33
jeblair2015-09-17 06:13:35,157 INFO nodepool.DiskImageBuilderThread: Running disk-image-create -x -t qcow2,vhd,raw --no-tmpfs --qemu-img-options 'compat=0.10' -o /opt/nodepool_dib/ubuntu-trusty-1442470415 ubuntu-minimal vm simple-init openstack-repos nodepool-base node-devstack cache-bindep growroot16:33
*** jordanP has quit IRC16:33
*** HeOS has quit IRC16:33
mordredjeblair: btw - it's not a problem, but I believe we can rmeove --qemu-img-options 'compat=0.10' now16:34
jeblairjust so we're clear -- i believe i've known what the problem is for 15 minutes now...16:34
mordredok16:34
mordredcool16:34
greghaynesok, so for some reason the row for that image isnt going to READY16:34
*** sdake has joined #openstack-infra16:34
jeblairno, the problem is the name mismatch.  let me work on a fix by way of explanation.  :)16:34
greghayneshaha, ok16:34
*** jistr has quit IRC16:35
* greghaynes wonders if by mismatch you mean u-t vs u-t-c16:35
greghaynesor is that just the u-t dib line16:36
jeblairgreghaynes: ut/utc16:36
*** david-lyle has quit IRC16:36
*** kiran-r has joined #openstack-infra16:37
*** ifarkas has joined #openstack-infra16:37
openstackgerritOpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements  https://review.openstack.org/22460616:38
*** sdake_ has quit IRC16:38
*** fedexo has joined #openstack-infra16:41
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from global requirements  https://review.openstack.org/22468516:43
clarkbmordred we dont need compat because hpcloud upgraded?16:43
*** glauco has joined #openstack-infra16:43
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Correct missing DIB image detection  https://review.openstack.org/22482116:43
*** hyakuhei has joined #openstack-infra16:43
jeblairgreghaynes, clarkb: ^ let me know what you think of that16:43
jeblair(meanwhile, i'll correct the exception too)16:44
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Initial support for graph panel  https://review.openstack.org/22481116:44
samueldmqhello, could I get a couple of extra eyes on https://review.openstack.org/#/c/224103/ ?16:45
samueldmqit has a +2 from fungi already, and it fixes a regression in our etherpad.o.o16:45
clarkbjeblair seems right, dsvm job will test it doesnt  break old behavior16:45
clarkbjeblair maybe we should update that job to use mismatched names?16:46
mordredclarkb: no, we don't need it because it's now just what dib does16:46
clarkbyou want to look in devstack/ and tools/check_devstack something or other to do that16:46
*** ericksonsantos has joined #openstack-infra16:46
clarkbmordred ah16:46
*** david-lyle has joined #openstack-infra16:47
*** DericHorn-HP has quit IRC16:47
mordredjeblair: looks good16:48
*** mtanino has joined #openstack-infra16:48
pabelangertimrc_: some new grafyaml reviews up! Adds the graph panels support16:49
*** glauco has quit IRC16:49
mordredpabelanger: ooo16:49
timrc_pabelanger, Cool, I'll review today if my day job doesn't keep getting in the way :)16:49
timrc_pabelanger, I'm really looking forward to having grafyaml deployed because turtles... I  mean metrics.16:50
*** timrc_ is now known as timrc16:50
greghaynesjeblair: have you retried deploying the hp pypi since last 401 unauthorized? (I am trying to kind of liason with support, curious what status is)16:50
*** derekh has quit IRC16:50
pabelangertimrc: Ya, going to start adding some of my local dashboards into project config.  Then deal with getting grafyaml running upstream16:51
*** glauco has joined #openstack-infra16:51
*** zeih has joined #openstack-infra16:51
greghaynesjeblair: They say they have reactivated services since you got that 401 so might be better now...16:51
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Correct exception when waiting for image build  https://review.openstack.org/22482316:51
greghaynesjeblair: also, awesome explanation of nodepool abstractions ;)16:52
jeblairgreghaynes: yay!16:52
*** unicell has joined #openstack-infra16:52
jeblair+----+------+--------+------------+-------------+----------+16:52
jeblair| ID | Name | Status | Task State | Power State | Networks |16:52
jeblair+----+------+--------+------------+-------------+----------+16:52
jeblair+----+------+--------+------------+-------------+----------+16:52
jeblairnova list works ^16:52
jeblair:|16:52
zaromorning16:53
clarkbactually non dsvm is sufficient since it wasnt related to dib itself16:53
greghaynesjeblair: is that supposed to be empty?16:53
clarkbjeblair thats not what nova list should look like16:53
*** Swami has quit IRC16:53
jeblairgreghaynes: i mean, ideally no.16:53
*** lucasagomes has quit IRC16:53
jeblairgreghaynes: depends on whether we expect that they have deleted the server.16:53
jeblairgreghaynes: (and the other nodepool thing is here: https://review.openstack.org/224823 )16:54
*** changbl has joined #openstack-infra16:55
mordredjeblair: well, since you're already working on trusty-ifying pypi mirrors, maybe this is a good time to make a shiny new trusty pypi mirror there too16:55
greghaynesjeblair: yep, everything was delted16:55
*** dramalho has joined #openstack-infra16:55
jeblairgreghaynes: then everything is working as expected!16:55
*** zeih has quit IRC16:55
d0ugalI've been added to https://review.openstack.org/#/admin/groups/190,members but it hasn't impacted my account. Wondering if I have somehow managed to make two gerrit accounts for dougal@redhat.com16:55
clarkbif we have to update dns anyeays lets switch to iad and turn on hpcloud?16:56
*** salv-orlando has quit IRC16:56
*** bltavares has joined #openstack-infra16:56
clarkblower the ttl too so a seitch to new host is quicker16:56
*** DericHorn-HP has joined #openstack-infra16:56
timrcpabelanger, Cool, I'll contribute some of our stuff.16:56
timrcI think we're going to get some really need data if/when my work merges to submit metrics from zuul by node label.16:56
*** fawadkhaliq has joined #openstack-infra16:57
*** _kiran_ has joined #openstack-infra16:57
timrcAt least that'll be true downstream where we're particularly resource-constrained.16:57
timrcneat* not need16:57
*** nadya has quit IRC16:57
jeblairtimrc: we're not particularly flush upstream at the moment16:57
zarowaynr: ping16:58
*** serg_melikyan has joined #openstack-infra16:58
timrcjeblair, ;-)16:59
jeblairmordred, clarkb: i'll go ahead and launch a trusty hpcloud node; it can sync while i hack out the puppet bits16:59
*** kiran-r has quit IRC16:59
clarkbok16:59
jeblairclarkb: are we going to need to ask hpcloud to let us use the internet again?16:59
clarkbjeblair the internet?17:00
jeblairclarkb: security groups17:00
mordredjeblair: run "nova secgroup-list-rules"17:00
clarkboh yes you need to delete the rules and add a couple to open everything17:00
jeblair| IP Protocol | From Port | To Port | IP Range | Source Group |17:01
jeblair+-------------+-----------+---------+----------+--------------+17:01
jeblair|             |           |         |          | default      |17:01
jeblair|             |           |         |          | default      |17:01
jeblairi don't know if that's good or bad17:01
*** sergmelikyan has quit IRC17:01
clarkbthas bad17:01
clarkbsays only your hosts can talk to your hosts17:01
jeblairclarkb: any chance you want to fix that while i launch the node?17:02
clarkbI can in a few yes17:02
jeblairclarkb: cool, no rush i think :)17:02
mordredjeblair: nova secgroup-add-rule default tcp -1 -1 0.0.0.0/017:02
mordredjeblair: want me to go do it?17:02
jeblairmordred: yeah, cause copy/paste that didn't work17:02
mordredawesome17:02
* mordred on it17:02
jeblairclarkb: you're off the hook :)17:02
jeblairoh, what flavor did we use for this?17:03
jeblairi'll check cacti17:03
*** david-lyle has quit IRC17:03
clarkbdont forget udp and icmp17:04
mordredyah17:04
clarkband delete the old rulrs17:04
*** harlowja has joined #openstack-infra17:04
jeblairclarkb, mordred: considering we have zero cacti data for it, i think we _did_ for get udp last time.17:04
clarkbsince thodr are the ones that break hpcloud17:04
jeblairanyone have a guess as to what flavor we should use?17:04
clarkbjeblair woo check rax?17:04
clarkbit should line up flavor wise and had a 300gb volume iirc17:05
jeblairclarkb: well, i was thinking in terms of ram and cpu... but probably we actually need to size it by ephemeral disk, right?17:05
*** salv-orlando has joined #openstack-infra17:05
clarkbjeblair no we use cinder volumes17:05
mordredjeblair: I'll have it in a saved inventory file - one sec17:05
clarkbso ephemeral disk is not used17:05
jeblairclarkb: we use cinder on hpcloud?17:05
*** dprince has quit IRC17:06
*** dizquierdo has quit IRC17:06
mordredoh - no I won't - I did not rsync my cache dir17:06
clarkbjeblair ya17:07
*** abitha has joined #openstack-infra17:07
*** dprince has joined #openstack-infra17:08
*** serg_melikyan has quit IRC17:08
*** sergmelikyan has joined #openstack-infra17:09
*** AJaeger has joined #openstack-infra17:09
*** trown is now known as trown|lunch17:10
mordredclarkb: http://paste.openstack.org/show/466589/17:10
*** romcheg has quit IRC17:10
jeblairclarkb, mordred: then i'm all ears for what we think the flavor should be17:10
openstackgerritPaul Michali proposed openstack-infra/project-config: Move VPN Functional tests to check queue  https://review.openstack.org/22304917:10
jeblairlooks like we use 4g machines on rax, with very little cpu17:11
clarkbmordred delete the default group rules we break hpcloud with those17:11
clarkbwhy they are there by default is beyond me17:11
jeblairshould we do 8g+4cpu on hpcloud then?17:11
clarkbsure17:12
mordredclarkb: ok. I believe we're good on the default sec group now17:13
*** romcheg has joined #openstack-infra17:14
jeblair/dev/xvdb       197G  188G     0 100% /srv17:16
jeblairi think the rax mirrors may be about full.17:16
*** angdraug has quit IRC17:17
mordredyay!17:17
*** fedexo has quit IRC17:17
jeblairmy stack is too deep; if someone else could take that, that would be great17:17
jeblairerm17:18
jeblairwe don't seem to be using lvm there17:18
jeblairwhy in the world would we not do that17:18
mordredthat seems like a bad choice17:18
jeblairit looks like the launch-node volume stuff doesn't support lvm... it just does what was done on the pypi mirrors...17:19
*** _kiran_ has quit IRC17:19
doug-fishis there a spec or similar doc for Zanata integration into stackalytics?17:19
*** tonytan4ever has quit IRC17:20
*** ihrachys has quit IRC17:20
pleia2doug-fish: I don't remember if it was actually part of the spec, I think it may have just been a bug with the zanata team directly17:20
openstackgerritTetiana Lashchova proposed openstack-infra/project-config: Add translation-jobs for python-heatclient  https://review.openstack.org/22483217:20
jeblairmordred, pleia2, fungi, jhesketh, clarkb, SergeyLukjanov:  all of the rax pypi mirrors need to have lvm volumes created and attached, and the bandersnatch mirror moved to them, and the existing cinder volume deleted.  i'm looking for a volunteer.17:20
pleia2doug-fish: yeah, not in the spec http://specs.openstack.org/openstack-infra/infra-specs/specs/migrate_to_zanata.html since it wasn't vital to launching, it was something Daisy said we could tackle next cycle17:21
pleia2doug-fish: Daisy would be the best one to ask re: status of her work with the Zanata folks and statistics17:21
doug-fishpleia2: cool thank - that's not a surprise to me, but I wanted to see what was planned for next cycle17:21
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add grafana dashboard for status.o.o/zuul  https://review.openstack.org/22483417:21
pabelangertimrc: mordred ^ example grafyaml dashboard for zuul status17:22
pleia2doug-fish: yeah, definitely, with i18n becoming all official we'll need a way to track ATC status for translations17:22
mordredjeblair: I'm close to end of day and have a phone call coming up - I cannot do it right now - I can work on it tomorrow though17:22
greghaynes"tomorrow"17:23
doug-fishpleia2: my concern-of-the-day is how email addresses/Zanata ids get associated with companies for rolling up contributions17:23
timrcpabelanger, What needs to happen to get grafyaml into place?17:23
timrcpabelanger, Last I checked you had your dashboard change as WIP/17:23
timrc?17:23
doug-fishwell, that's one of my concerns-of-the-day17:23
greghaynesAKA tonight :)17:23
*** mriedem has quit IRC17:23
pabelangertimrc: not much, I need to build puppet-grafyaml and import it17:23
pabelangertimrc: then, add test coverage for datasource17:24
pabelangerand some sort of caching like JJB17:24
pleia2doug-fish: openstackid is used for login, which is what we use on openstack.org and want to eventually use for everything, so we're kind of ahead of everyone else when it comes to using non-launchpad federated login, if there isn't mapping now, there will be soon17:24
jeblairmordred: hopefully someone else will volunteer to do it sooner, as 0 bytes free makes it somewhat urgent (thoug, we're probably just not breaking openstack as fast as normal since the mirror is effectively frozen)17:25
*** ybathia has joined #openstack-infra17:25
doug-fishpleia2: I think you are telling me that want I want will work - that is if we have people contributing translations on behalf of IBM that those will get rolled up into the IBM totals that we would see on stackalytics, even if they don't have IBM email ids.17:26
*** sergmelikyan has quit IRC17:26
*** sergmelikyan has joined #openstack-infra17:27
*** mriedem has joined #openstack-infra17:27
timrcpabelanger, Any thoughts on how to organize dashboards?  E.g. do we want a dashboard that lists other dashboards that are associate with certain tags?17:27
doug-fishpleia2: my other concern-of-the-day that I'd like to share is that we are uploading all of the translations we'd like to contribute to the translation-dev server today (to make sure we aren't going to break the production server)17:27
jeblairnovaclient.exceptions.BadRequest: No nw_info cache associated with instance (HTTP 400) (Request-ID: req-8b664234-bc8a-442a-bcec-1c0b6017883b)17:28
jeblairwhat in the WORLD does that mean ^?17:28
timrcpabelanger, I really hate grafana's opinion on what an organization is.  The fact grafana only allows one anonymous organization makes logically segregating dashboards difficult.17:28
pleia2doug-fish: yeah, just like right now with launchpad, you don't need to be using a company email address, we have a file that maps email addresses to companies, we'll definitely have something like that for openstackid too17:29
greghaynesjeblair: https://bugs.launchpad.net/nova/+bug/1249065 huh.17:29
openstackLaunchpad bug 1249065 in OpenStack Compute (nova) "Nova throws 400 when attempting to add floating ip (instance.info_cache.network_info is empty)" [Medium,Fix released] - Assigned to Matt Riedemann (mriedem)17:29
pabelangertimrc: Ya, I haven't done much work with orgs. For me, I've just used the default one to create dashboards.  However, I am just a single person :D17:29
doug-fishpleia2: excellent! thanks!17:29
pleia2doug-fish: that's fine re: uploading translations to the -dev server, there should be a plan for migrating all the translations to the production server at some time, AJaeger may be a good one to talk to about that, we don't want to get in the way of the automated scripts17:30
greghaynesjeblair: looks like its a race17:30
*** tonytan4ever has joined #openstack-infra17:30
timrcpabelanger, When you start getting a lot of dashboards it can become quite chaotic.17:31
* jeblair launches again17:31
doug-fishpleia2: we actually talked through that in the i18n meeting today. We are going to start with separate branches (just like we are doing on -dev today), then the language coordinators can merge when (if?) they are comfortable with a contribution17:31
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: Add devstack-plugin-bdd to project config  https://review.openstack.org/22473817:31
AJaegerpleia2: sorry, can't help with that. I don't think our scripts touch the -dev server - and if it's a separate version, it's no problem either.17:31
pabelangertimrc: Ya. I am hoping our git workflow will help with some of that17:31
*** ifarkas has quit IRC17:31
pleia2AJaeger: not on the dev server, when we move into production17:32
pleia2s/we/the translations17:32
pleia2don't want to upload files over existing translations or anything17:32
pleia2but something like a separate branch should work17:32
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: Add devstack-plugin-bdd to project config  https://review.openstack.org/22473817:32
fungiokay, sorry, back at computer now17:33
fungicatching up17:33
*** Flcn has joined #openstack-infra17:34
mriedemgreghaynes: jeblair: yeah it's a race17:34
jeblairmriedem: i'm 0/2 so far :(17:34
mriedemshould be squashed in liberty as of last week17:34
jeblair0/317:34
greghaynesjeblair: youre winning the race!17:34
mriedemjeblair: nova-net right?17:34
fungii see that pypi.region-b.geo-1.openstack.org still does not respond to ping but our creds are working there again (albeit with empty nova list so i guess the account and contents were baleeted)17:34
*** ivar-lazzaro has joined #openstack-infra17:34
jeblairmriedem: erm, whatever hpcloud runs in production17:34
mriedemanyone know what that is? i assume neutron17:35
jeblairmriedem: but yeah, i'm at least talking to nova, not neutron17:35
mriedemoh17:35
mriedemdo we know what version?17:35
*** abregman_ has quit IRC17:35
*** austin81 has joined #openstack-infra17:35
jeblairmriedem: oh, i think they use neutron, so it must be proxied.17:35
AJaegerpleia2: easiest looks like a script that downloads current translation and IBM translation, merges them with giving current priority and upload again.17:35
jeblairmriedem: and don't know version info17:36
AJaegerThere's still a small race that people edit one of those strings at that time but that can always happen.17:36
mriedemjeblair: ok, i'm intimately familiar with the nova-net race,17:36
mriedemi feel like there was a related neutron api fix for that same error in nova at some point, i can poke17:36
jeblairmriedem, greghaynes: i've hit it 4 out of 4 attempts so far in production hpcloud17:36
mriedemmestery: do you know if hpcloud runs with neutron?17:37
jeblairgreghaynes: i think it might be time to start talking to the hpcloud folks17:37
mriedemand if so, what eversion of nova/neutron?17:37
*** bltavares has quit IRC17:37
greghaynesjeblair: yep, one sec17:37
mesterymriedem: Aye aye, it indeed does, /cc haleyb carl_baldwin17:37
AJaegerpleia2: what's up with all the python-Xclient projects that now want translations?17:37
jeblairmriedem: if i sleep before adding the floating ip will it work? :)17:38
mriedemAJaeger: zanata stuff that happened this week right?17:38
pleia2AJaeger: I don't know, more than just keystone?17:38
mriedemjeblair: it's not actually adding the floating IP that's the problem, at least it wasn't in the nova-net case17:38
mriedemjeblair: might be different for neutron api17:38
pleia2AJaeger: I thought the python-Xclients were mostly deprecated in favor of the openstack one x_x17:38
mriedempleia2: that's going to be a long time coming17:38
AJaegerpleia2: https://review.openstack.org/224832 heat and https://review.openstack.org/224222 nova17:38
pleia2mriedem: aha17:38
pleia2AJaeger: I see17:38
*** achanda has joined #openstack-infra17:39
pleia2AJaeger: yeah, I have no idea what triggered this17:39
mriedemAJaeger: oh you mean why are there now a flood of requests for those17:39
AJaegermriedem: yes17:39
pleia2mriedem: yeah17:39
mriedemAJaeger: it's because we've been translating strings in the clients but not actually having translations for them17:39
AJaegermriedem: three in a week - and before that now....17:39
mriedemlike people were literally pushing changes to _() things17:39
pleia2just the one before that17:39
AJaegermriedem: marking strings, not translating ;)17:39
mriedembut it was useless since we didn't have the actual translations in tree17:39
mriedemyeah17:39
AJaegermriedem: the translators have limited resources and have to priotize.17:40
greghaynesjeblair: this is happening when you try to attach a FIP to an instance?17:40
mriedemAJaeger: pleia2: http://lists.openstack.org/pipermail/openstack-dev/2015-September/073698.html17:40
jeblairgreghaynes: yes; fip creation works17:40
AJaegermriedem, I suggest you talk with Daisy. I would see as prios: Horizon and friends, openstack client - and then the python clients, and then server projects (let's leave docs out of this).17:41
doug-fishmriedem: And Horizon is and should be the top priority.  :-P17:41
mriedemhonestly i don't care about priorities17:41
mriedemi just thought it was dumb that we were marking strings in a project that didn't actually have translation support setup in infra17:41
pleia2well, you are adding to the burden of the translations team17:41
haleybjeblair: hpcloud having issues ?  just reading scrollback17:41
pleia2so talking to them would be good17:41
mriedempleia2: that's why i suggested in the ML to just stop marking strings in novaclient and pretending like it was helpful17:42
jeblairhaleyb: yep, seeing novaclient.exceptions.BadRequest: No nw_info cache associated with instance (HTTP 400) (Request-ID: req-f80b8ab6-fc88-4491-9ea6-fefe9a333691)17:42
pleia2mriedem: yeah, that's fair17:42
mriedemmy priority right now is figuring out what jeblair is hitting :)17:42
*** austin81 has left #openstack-infra17:42
jeblairhaleyb: which apparently is bug 124906517:42
openstackbug 1249065 in OpenStack Compute (nova) "Nova throws 400 when attempting to add floating ip (instance.info_cache.network_info is empty)" [Medium,Fix released] https://launchpad.net/bugs/1249065 - Assigned to Matt Riedemann (mriedem)17:42
mriedemb/c i think it's fixed17:42
mriedemjeblair: haleyb: https://review.openstack.org/#/c/186315/17:43
AJaegermriedem: let me write an email to the i18n list17:43
greghaynesmriedem: its possible they are just running old code17:43
mriedemhttps://review.openstack.org/#/q/I3b9f60a2c4aea904f39d6914c8f35c4d682a6b3a,n,z goes back to stable/juno17:43
fungiand now i think i'm fully caught up, at least as much as skimming scrollback can accomplish17:43
mriedemi'd expect hpcloud to be at least at kilo17:43
*** rm_work has quit IRC17:43
jeblairfungi: any chance you want to take on the rax cinder work?17:43
mriedemthat might not be the issue, but it's related17:43
fungijeblair: mordred: pleia2: jhesketh: clarkb: SergeyLukjanov: i'll start working on cinder volumes for our rax servers17:44
fungijeblair: ;)17:44
jeblairfungi: awesome :)17:44
*** austin81 has joined #openstack-infra17:44
fungiyeah, i didn't see anyone else volunteer anyway17:44
greghaynesjeblair: pinged in support chat... hopefully get some response on it17:44
*** vgridnev has joined #openstack-infra17:45
*** rm_work has joined #openstack-infra17:45
fungiwow, yeah. we've got some pretty thoroughly full filesystems on our pypi mirrors. i guess i should just go with 0.5tb cinder volumes for each and use lvm this time around17:46
fungithis will... not be fast17:46
jeblairfungi: yeah, that's my plan with hpcloud17:46
fungiquite a bit of data to rsync17:46
*** dahoo has joined #openstack-infra17:46
fungiat least i can set them going in parallel17:46
*** dahoo is now known as jerryz17:46
*** daenney has quit IRC17:47
haleybjeblair: i work on the hpcloud neutron team, so don't know if that fix is in the queue17:47
openstackgerritRyan Carey proposed openstack-infra/jenkins-job-builder: Add support for Artifactory plugin  https://review.openstack.org/17393217:47
fungiwill almost certainly need to blow away the bandersnatch state and resync it once that's done too17:47
*** sergmelikyan has quit IRC17:47
*** julim has quit IRC17:48
*** julim has joined #openstack-infra17:48
*** bltavares has joined #openstack-infra17:48
*** amuller is now known as amuller_afk17:49
*** sergmelikyan has joined #openstack-infra17:50
FlcnHello. My merge request was merged to master 2 days ago, and PTL said that i can still receive ATC pass if write here. Somebody know were i need to request them? Thanks.17:50
*** ociuhandu has quit IRC17:50
fungiFlcn: contact summitreg@openstack.org and see if they can issue one to you17:51
greghaynesjeblair: do you have an instance-id for one of the nodes?17:51
fungiFlcn: the infra team doesn't actually handle those, i just happen to run the scripts that send the mass mailings of them because i also work for the foundation17:51
jeblairgreghaynes: i'll generate one17:51
Flcn@funigi Thanks17:51
jeblairgreghaynes: | da8d0813-1c62-4b8c-bfb4-5a87072e3f8e | pypi.region-b.geo-1.openstack.org | ACTIVE | -          | Running     |          |17:52
jeblairgreghaynes: novaclient.exceptions.BadRequest: No nw_info cache associated with instance (HTTP 400) (Request-ID: req-5cc635c6-5ff8-4306-9373-1f416262c81c)17:52
jeblairgreghaynes: there's a matching instance-id and request-id for the failed req17:52
fungiia shub-pypi, ia!17:52
*** sergmelikyan has quit IRC17:52
*** dprince has quit IRC17:53
greghaynesjeblair: ty17:53
*** vgridnev_ has joined #openstack-infra17:53
pleia2jumping on a call, will bbiab17:54
fungiit's unfortunate nova volume-attach doesn't take volume names17:54
fungisilly extra lookup17:54
openstackgerritJames E. Blair proposed openstack-infra/system-config: Revert "Add support to launch node for attaching volumes"  https://review.openstack.org/22484817:55
*** vgridnev has quit IRC17:55
fungiyeah, that's simply created extra work in the end17:55
openstackgerritJames E. Blair proposed openstack-infra/system-config: Disable volume option until it is cinderified.  https://review.openstack.org/22484917:56
jeblairor the less intrusive option ^17:57
greghaynesjeblair: support says instance isnt getting a private ip either17:57
greghaynesjeblair: do you have a nova command I can give to support for creating the instances17:57
clarkbmaybe we lack a network and router?17:57
clarkbnova show-net or something17:58
jeblairnova net-list is 40417:58
clarkboh you may need to use neutronclient17:59
*** SumitNaiksatam has joined #openstack-infra17:59
clarkbbecause some things are proxied but notbothers just to be confusing17:59
jeblair| id                                   | name    | subnets                               |18:00
jeblair+--------------------------------------+---------+---------------------------------------+18:00
jeblair| 7da74520-9d5e-427b-a508-213c84e69616 | Ext-Net | e9440419-d276-4c3d-9c18-7991b77e2465  |18:00
jeblair|                                      |         | ef3bde66-8ac4-4356-8ab4-099519ba218a  |18:00
*** dkranz has quit IRC18:00
jeblairneutron router-list is empty; like not even a table18:01
jeblairgreghaynes: let me see if i can recreate the process with nova commands18:01
*** Flcn has quit IRC18:01
*** HeOS has joined #openstack-infra18:01
*** vgridnev_ has quit IRC18:02
*** trown|lunch is now known as trown18:03
haleybjeblair: do the tools create networks on the fly?  or is there one default one ?18:03
*** bhunter71 has quit IRC18:04
*** pvaneck has joined #openstack-infra18:04
jeblairhaleyb: our tools don't create networks; we're counting on a default one.  that makes me think that there is a default one, though i don't know why there are 2 subnets, and i don't understand the router situation.18:04
greghaynesjeblair: Yea, so sounds like our network config is not like it should be - thats something we set up by hand?18:04
*** doug-fi__ has joined #openstack-infra18:04
*** doug-fish has quit IRC18:04
greghaynesoh, heh18:04
jeblairgreghaynes: apparently?  i am unaware of any documentation on the subject.18:04
*** jamielennox is now known as jamielennox|away18:05
haleybjeblair: we have two public networks, they show up since the floaters need them18:05
*** dprince has joined #openstack-infra18:05
*** kgiusti has quit IRC18:05
haleybthat command should have showed  a private network as well18:05
*** kgiusti has joined #openstack-infra18:05
jeblairhaleyb: that sounds like it might explain why we're not getting a private ip too.18:06
*** doug-fi__ has quit IRC18:06
*** nadya has joined #openstack-infra18:07
*** e0ne has joined #openstack-infra18:07
haleybsounds like something deleted your neutron pieces18:07
clarkbI rhink the deletion broke us18:07
clarkband undelete didnt populate with defaults18:07
clarkbso we need to manually make a subnet, a network, a router and hook them up18:07
clarkb(for anyone following along this is why we argue against all this complicated stuff in neutron)18:08
jeblairi have no idea how to do this18:08
jeblairi am going to log into the gui and click on things until it looks like it might work18:08
*** jsavak has quit IRC18:08
*** bltavares has quit IRC18:08
clarkbyes gui is best bet18:08
clarkbhorizon makes it easier18:08
greghaynes:(18:09
greghaynesif we say that...18:09
*** bltavares has joined #openstack-infra18:09
greghayneswho doesnt?18:09
openstackgerritAlex Schultz proposed stackforge/gertty: Make permalink clickable  https://review.openstack.org/22485118:09
jeblairi'm told we need a private network, so i'm going to make a network called "private"18:10
jeblairthere is a checkbox labeled "Admin State".  it is checked.  I don't know what it means.18:10
jeblairi will leave it checked18:10
*** hyakuhei has quit IRC18:10
clarkbjeblair in the other accounr is accountname-networkX iirc18:10
haleybat the same time you make it (in horizon) you can also create a subnet18:10
*** doug-fish has joined #openstack-infra18:10
jeblairdo i want a subnet?18:10
clarkbyes its required18:11
clarkbI would do a /2418:11
* haleyb opens a horizon window on his hpcloud account to help if necessary18:11
clarkbsince a router cant handle more than that many hosts sanely18:11
*** hyakuhei has joined #openstack-infra18:11
clarkband 10.0.0.0/24 is probably fine18:11
*** pelix has quit IRC18:11
jeblairenable dhcp?18:11
haleybyes18:12
clarkbyes18:12
clarkband probably set google dns18:12
jeblairdo i need to fill in the allocation pools, or will it just use the whole range if i leave it blank?18:12
*** sabeen1 has joined #openstack-infra18:12
clarkbthough puppet will anyways18:12
clarkbit will do the right thing if blank18:12
clarkbreserves first for router and second for dnsmaq and everything else is ours iirc18:13
jeblairoh, "admin state" means "up or down"18:13
fungijeblair: mordred: pleia2: jhesketh: clarkb: SergeyLukjanov: rsync is running under a root-owned screen session on each of the rax pypi servers. i'll check on them periodically and swap them into service once completed18:13
jeblairfungi: thank you!18:13
fungia pleasure as always18:13
*** jsavak has joined #openstack-infra18:14
*** hyakuhei has quit IRC18:14
jeblairhrm, doesn't show up in neutron net-list18:14
*** tonytan4ever has quit IRC18:15
jeblairoh well.  i'll make a router now.18:15
*** dkranz has joined #openstack-infra18:15
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Remove Babel bits  https://review.openstack.org/22485518:16
haleybjeblair: there are two datacenters for hpcloud, perhaps your horizon is pointing at one, commands running against the other?18:16
*** e0ne has quit IRC18:17
jeblairhaleyb: ah, that was it18:17
*** sabeen3 has joined #openstack-infra18:17
haleybi don't know if infra uses both18:17
openstackgerritDanilo Ramalho proposed openstack-infra/puppet-logstash: Add acceptance tests for puppet-logstash.  https://review.openstack.org/22485818:18
*** bhunter71 has joined #openstack-infra18:18
haleybyour previous net-list was showing our East datacenter18:18
fungihaleyb: we've been using "region b" a.k.a. "usa east 1.1" i think18:18
fungihard to know what to call it since hpcloud has renamed it at least a couple times18:19
*** AJaeger has quit IRC18:19
*** sdake has quit IRC18:19
*** sabeen1 has quit IRC18:19
jeblairokay switching to east and repeating18:19
haleybjeblair: so you will need to go create a network in East18:19
jeblairapparently there is an ipv6 subnet option?18:20
haleybfungi: horizon calls it US East and West, so I'm sticking with that :)18:20
haleybjeblair: no, but horizon does show it18:20
fungihaleyb: thanks, since we generally never touch the web interface it's hard to know18:20
*** ociuhandu has joined #openstack-infra18:20
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Bump pbr requirement to >= 1.3  https://review.openstack.org/22486018:20
*** e0ne has joined #openstack-infra18:21
jeblairhaleyb: yeah, makes sense -- though the horizon/api disconnect is why i missed that (our cli tools use region-b.geo-1, and i can never remember if that's east or west)18:21
*** jsavak has quit IRC18:21
haleybfungi: the region name in an openrc file does have region-b, so there's your confusion18:22
jeblairyay, shows up in net-list now18:22
*** jsavak has joined #openstack-infra18:22
*** sdake has joined #openstack-infra18:23
fungihaleyb: well, it was also "1.1 beta" for a while when we first started using it, and since that was only in the "new" (east) region at the time the cardinal direction tended not to get mentioned18:23
*** doug-fish has quit IRC18:23
*** doug-fish has joined #openstack-infra18:23
*** AJaeger has joined #openstack-infra18:24
*** alexsyip has joined #openstack-infra18:25
*** hyakuhei has joined #openstack-infra18:25
*** vmbrasseur_BIAB is now known as vmb18:25
jeblairokay, our network topology looks like an H now18:25
openstackgerritColleen Murphy proposed openstack-infra/system-config: (WIP) Add compute class for Infra Cloud  https://review.openstack.org/21398018:26
openstackgerritColleen Murphy proposed openstack-infra/system-config: (WIP) Add controller class for Infra Cloud  https://review.openstack.org/20969818:26
haleybi think that's goodness, as it implies router has interface on Ext-Net18:26
*** hyakuhei has quit IRC18:26
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Simplify .gitignore  https://review.openstack.org/22486418:26
jeblairokay, launching again18:27
*** Zhongjun has quit IRC18:27
*** doug-fish has quit IRC18:27
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Drop pypy test coverage  https://review.openstack.org/22486518:28
jeblairneat, now it's an H with a thing hanging off the right side :)18:28
jeblairi think we're making progress18:28
*** doug-fish has joined #openstack-infra18:28
*** doug-fis_ has joined #openstack-infra18:29
jeblairmriedem: it looks like that error message can also mean "your cloud provider forgot to reset your default network topology after reactivating your deleted account"18:30
jeblairhrm18:30
jeblairthe instance is up, but i'm unable to access it over the internet18:30
*** ihrachys has joined #openstack-infra18:30
jeblairi can see the console in horizon though18:30
*** doug-fi__ has joined #openstack-infra18:30
clarkbfip attached?18:31
*** mmedvede has quit IRC18:31
fungidefault security groups blocking everything?18:31
*** nadya has quit IRC18:31
jeblairclarkb:      10.0.0.3,  15.126.140.718:31
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Clean up tox.ini  https://review.openstack.org/22486618:31
jeblairALLOW IPv4 to 0.0.0.0/018:31
*** fawadkhaliq has quit IRC18:31
jeblairALLOW IPv4 from default18:31
clarkbneed to delete the default rule18:31
*** sdake has quit IRC18:31
*** fawadkhaliq has joined #openstack-infra18:31
* fungi assumes "default" means the local network in this case18:31
clarkbbut that shouldnt cause this18:32
mriedemjeblair: hmm, well, the nw_info not found thing means we literally don't have nw_info in the nova database for that instance18:32
mriedemjeblair: and nw_info should be stored on the instance after the instance is first created18:32
clarkbfungi it means any host in the same group18:32
*** fawadkhaliq has quit IRC18:32
fungirighg18:32
clarkbit breaks hpcloud18:32
mriedemthe race before was nw_info was reset/wiped out after booting the instance but before attaching the floating ip18:32
jeblairclarkb: yeah, the defaults are the only ingress rule18:32
jeblairlooks like mordred didn't complete the work18:32
jeblairwill fix in gui18:32
mriedemjeblair: did you find out what version of nova they are running?18:32
*** doug-fish has quit IRC18:32
*** Flcn has joined #openstack-infra18:33
*** hyakuhei has joined #openstack-infra18:33
*** sdake has joined #openstack-infra18:33
*** doug-fis_ has quit IRC18:33
*** sabeen3 has quit IRC18:34
*** hyakuhei has quit IRC18:34
*** sabeen1 has joined #openstack-infra18:34
*** doug-fi__ has quit IRC18:35
jeblairmriedem: no, and i don't think haleyb knows either18:35
clarkbjeblair maybe mordred got regions wrong too?18:35
*** Flcn has joined #openstack-infra18:36
jeblairclarkb: no, other region has ingress range=default18:36
clarkbhuh18:36
haleybmriedem: i don't know if that fix is in production, i pinged the nova team18:36
openstackgerritKevin Fox proposed openstack-infra/puppet-apps_site: Fix for Murano Liberty version issue.  https://review.openstack.org/22486918:37
*** dprince has quit IRC18:37
jeblairand puppet is running on the server18:37
fungii thought in the past we set the source on the ingress rule to 0.0.0.0/0 as well18:38
fungimaybe compare to the security groups on the openstackjenkins2 account's routers?18:38
jeblairfungi: s/router/security group/?18:38
fungioh, are security groups not per-router?18:39
jeblairthey don't seem to be?18:39
clarkbno they are per host18:39
clarkbyou can apply them to VMs independent of the router they sit on18:39
jeblairwow18:40
fungigot it. i'm constantly forgetting which openstack abstractions are per what other abstractions18:40
clarkbso for us we just say allow all icmp udp and tcp traffic wverywhere in the default group18:40
jeblairit's kind of a mess over there18:40
clarkbits 3 rules I think18:40
jeblairoh, west again, sorry, switching18:40
jeblairwe're east, right?18:40
clarkbyes18:40
jeblairokay, er, i'm sticking with it's a mess:18:40
*** Flcn has quit IRC18:42
jeblairopenstackjenkins2 has ingress: tcp/0-65535, tcp/22, icmp, udp/0-6553518:42
jeblairso 4 rules18:42
jeblairin openstackci, i made 1 ingress rule: any ipv4 protocol18:42
jeblairclarkb, fungi: did you end up changing dns for hpcloud?18:43
fungii did not. afaik we still have nodepool's max set to 0 there18:43
clarkbtcp/22 is unnecessary and no I did not change it18:44
clarkbbeen swamped with family home things this morning18:44
clarkbgoing to be at thing tonight which means I parent now18:45
jeblairfungi: do you want to change dns to point to iad, with a, say, 300 or 600 second timeout?18:45
*** mmedvede has joined #openstack-infra18:45
fungijeblair: sure, i'll do that now18:45
*** AJaeger has quit IRC18:46
fungiwe're getting pretty far behind18:46
jeblairfungi: and re-enable hpcloud?  we're still hours away from having a usable mirror and our queue is climbing18:46
mriedemhaleyb: ok18:46
fungii also suspect, based on the node graph, that we're failing to build much in rax (likely over quota or some other error there)18:46
haleybjeblair: ping me if you need more help with hpcloud/neutron, but need to step out for a little bit18:47
jeblairhaleyb: thanks, i think we're all good now.  thanks for your help!18:48
fungi"This First Generation server will be converted to a Next Generation server after October 6, 2015. You can choose to convert it yourself by soft rebooting the server."18:48
fungineat! rax is near-seamlessly migrating their old virtual servers to nova18:48
*** greghaynes has quit IRC18:49
*** ihrachys has quit IRC18:50
fungii can't remember what we're still serving from the old wiki server besides old-wiki.openstack.org and some long-abandoned redirects like nova.openstack.org18:50
fungimight should look into deleting that soonish if there's nothing of value remaining18:50
*** erikwilson has quit IRC18:51
*** jsavak has quit IRC18:51
jeblairfungi: i can never remember -- did we finish moving tarballs?18:51
*** zeih has joined #openstack-infra18:51
*** johnsom has left #openstack-infra18:52
fungiyep, that's on static.o.o now18:52
jeblairyes, apparently :)18:52
*** jsavak has joined #openstack-infra18:52
fungidns agrees!18:52
*** ihrachys has joined #openstack-infra18:52
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Pull tripleo related packages from delorean/current  https://review.openstack.org/22416818:53
*** ddieterly has joined #openstack-infra18:53
openstackgerritMerged openstack-infra/tripleo-ci: Stop building tuskar-ui-extras  https://review.openstack.org/22240718:54
fungiipv4 address for pypi.region-b.geo-1.o.o has been switched to match pypi.iad.o.o in dns and ttl set to 300s. the previous ttl was 3600s and so google's resolvers may still have it cached for a while18:54
*** ozamiatin has joined #openstack-infra18:55
*** burgerk__ has joined #openstack-infra18:55
fungisoliciting opinions on how long we should wait to spin up workers in hpcloud again18:55
*** greghaynes has joined #openstack-infra18:56
*** zeih has quit IRC18:56
jeblairfungi: probably the full hour :|18:56
*** ddieterly has quit IRC18:56
fungi'twas my assumption as well18:56
jeblairthe new hpcloud mirror is performing the initial bandersnatch sync18:57
fungithat'll be hours to complete still anyway18:57
openstackgerritKevin L. Mitchell proposed stackforge/python-jenkins: Allow setting the next build number  https://review.openstack.org/22427518:57
jeblairyup18:57
*** hyakuhei has joined #openstack-infra18:57
jeblairi've been in this chair for too many hours; i'm going to resume after an extended lunch18:57
fungirsync is currently on packages starting with capital C and seems to be going in posix sort order18:58
fungifor the rax pypi volume replacementds18:58
*** burgerk_ has quit IRC18:58
greghaynesjeblair: made a test case for your nodepool fix and I think theres a bug18:58
jeblairgreghaynes: awesome, thanks!18:59
*** tonytan4ever has joined #openstack-infra18:59
jeblairi'll start with that after lunch18:59
*** ccarrara has quit IRC18:59
*** amuller_afk is now known as amuller18:59
*** burgerk__ has quit IRC19:00
*** dprince has joined #openstack-infra19:00
*** glauco has quit IRC19:01
*** glauco has joined #openstack-infra19:01
*** erikwilson has joined #openstack-infra19:02
fungioh, the ordering on this rsync may be less normalized than i thought. really no clue, going by inode count we're at ~50% complete i think19:02
*** hyakuhei has quit IRC19:02
fungiwait, no. looked at wrong numbers. ~25%19:02
Clintdo not taunt happy fun ball19:02
*** erikwilson has quit IRC19:02
*** dzamboni has quit IRC19:02
*** abregman has joined #openstack-infra19:03
*** ccarrara has joined #openstack-infra19:04
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Stop building ironic-discoverd  https://review.openstack.org/22240819:06
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Remove support for mgt-master branches  https://review.openstack.org/22359919:06
fungihave to pop out again for a few minutes... brb19:08
*** devlaps has quit IRC19:08
*** dkranz has quit IRC19:08
*** austin81 has left #openstack-infra19:10
*** austin81 has joined #openstack-infra19:10
*** glauco has quit IRC19:11
*** bhunter71 has quit IRC19:11
*** david-lyle has joined #openstack-infra19:11
*** kushal has joined #openstack-infra19:12
*** glauco has joined #openstack-infra19:13
*** teran_ has quit IRC19:15
*** glauco has quit IRC19:16
*** olaph has quit IRC19:18
*** glauco has joined #openstack-infra19:19
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Pull tripleo related packages from delorean/current  https://review.openstack.org/22416819:20
*** glauco has quit IRC19:20
*** matrohon has joined #openstack-infra19:21
*** dkranz has joined #openstack-infra19:22
*** hdd has quit IRC19:22
*** hdd has joined #openstack-infra19:23
*** bswartz has joined #openstack-infra19:24
*** bhunter71 has joined #openstack-infra19:25
*** david-lyle has quit IRC19:25
greghaynesjeblair: I think (minus the issue I pointed out on review) your patch fixes builds, but uploads are also broken in a similar way :(19:25
* greghaynes -> lunch, bbiab19:26
*** claudiub has quit IRC19:27
*** ccarrara has quit IRC19:27
*** _skolekonov has quit IRC19:28
*** armax has quit IRC19:28
*** vmbrasseur is now known as vmb_work_AWAY19:31
*** vmb is now known as vmbrasseur19:33
*** ozamiatin has quit IRC19:33
*** jlebon has left #openstack-infra19:35
*** alexpilotti has quit IRC19:35
dims_zuul is in rough shape...19:36
clarkbdims_ we are only running on one cloud19:38
dims_clarkb: y i saw that float by19:38
*** Sukhdev has joined #openstack-infra19:39
*** ZZelle_ has joined #openstack-infra19:39
*** gordc has quit IRC19:39
*** pc_m has quit IRC19:42
openstackgerritColleen Murphy proposed openstack-infra/system-config: (WIP) Add compute class for Infra Cloud  https://review.openstack.org/21398019:42
openstackgerritColleen Murphy proposed openstack-infra/system-config: (WIP) Add controller class for Infra Cloud  https://review.openstack.org/20969819:42
EmilienMtrying to push a new puppet module for one week: https://review.openstack.org/#/c/222289/ - any chance to have a review please :-) ?19:45
*** sridhar_ram has quit IRC19:45
*** sdake has quit IRC19:46
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Summit Application API  https://review.openstack.org/22196419:47
*** Piet has quit IRC19:47
*** sdake has joined #openstack-infra19:48
fungiokay, back again19:49
*** derekh has joined #openstack-infra19:51
*** Sukhdev has quit IRC19:51
*** HeOS has quit IRC19:51
openstackgerritBen Nemec proposed openstack/diskimage-builder: Update rhel7 element readme  https://review.openstack.org/22235119:52
*** ldnunes has quit IRC19:52
*** Sukhdev_ has joined #openstack-infra19:53
*** ldnunes has joined #openstack-infra19:56
*** sdake has quit IRC19:57
pleia2hm, I don't remember where we landed on which projects require what goverance changes and the depends-on order (looking at EmilienM's new puppet module depending on a governance page)19:58
*** earlephilhower has quit IRC19:59
EmilienMpleia2: it has been one week now, TC should approve the change (non negative reviews)19:59
EmilienMpleia2: I just want to create puppet-aodh :)19:59
EmilienMwhich is required to run ceilometer19:59
pleia2EmilienM: well, this won't land until the governance change is approved19:59
*** sdake has joined #openstack-infra20:00
*** ihrachys has quit IRC20:00
EmilienMpleia2: yep... I asked ttx if we could approve the gov change - will see tomorrow20:00
pleia2the change looks fine, I just need a refresher on how we handle our depends here20:01
*** fitoduarte has quit IRC20:04
*** fitoduarte has joined #openstack-infra20:05
*** zns_ has quit IRC20:06
*** Sukhdev_ has quit IRC20:06
*** Piet has joined #openstack-infra20:07
*** teran has joined #openstack-infra20:09
*** HeOS has joined #openstack-infra20:09
*** zns has joined #openstack-infra20:11
*** Piet has quit IRC20:12
*** Piet has joined #openstack-infra20:13
*** sabeen1 has quit IRC20:13
*** Piet has quit IRC20:13
jeblairEmilienM, pleia2: the governance change can depend on the repo creation (ie, you can reverse it)20:14
EmilienMjeblair: do you want me to change something?20:14
jeblairsimple additions of repos to existing projects are assumed okay with ptl ack20:14
jeblairEmilienM: up to you -- whether or not you want to wait :)20:14
EmilienMjeblair: I don't20:14
EmilienMlet's change it20:14
jeblairk20:14
pleia2thanks jeblair20:14
*** bhunter71 has quit IRC20:15
openstackgerritsebastian marcet proposed openstack-infra/openstackid: OIDC - OpenId Connect Implementation  https://review.openstack.org/19845820:15
jeblairhpcloud bandersnatch is at 28G (out of a presumed >= 200G)20:15
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Create puppet-aodh repository  https://review.openstack.org/22228920:15
jeblairgreghaynes: you said you wrote a test?20:16
greghaynesjeblair: Yea, although I havent gotten it to pass because hitting other fails20:17
greghaynesjeblair: I can push it up as depndent20:17
*** dramalho has quit IRC20:17
*** lucasagomes has joined #openstack-infra20:17
*** glauco has joined #openstack-infra20:18
*** teran has quit IRC20:18
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Correct exception when waiting for image build  https://review.openstack.org/22482320:19
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Correct missing DIB image detection  https://review.openstack.org/22482120:19
jeblairgreghaynes: cool, thx.  and i agree with your -1 :)20:19
*** fifieldt has quit IRC20:19
jeblairfungi, clarkb: er, it looks like we run bandersnatch in a 30m timeout...20:20
*** fifieldt has joined #openstack-infra20:21
jeblairi think the initial syncs are going to be hard pressed to work under those circumstances20:21
jeblairthat's probably why gra1 is only at 76G20:21
fungijeblair: yep, it used to have a tendency to hang, so starting a fresh sync means don't do it under cron20:21
openstackgerritColleen Murphy proposed openstack-infra/system-config: (WIP) Add compute class for Infra Cloud  https://review.openstack.org/21398020:21
openstackgerritColleen Murphy proposed openstack-infra/system-config: (WIP) Add controller class for Infra Cloud  https://review.openstack.org/20969820:21
*** derekh has quit IRC20:21
fungithe timeouts were a safeguard against bandersnatch getting stuck in a livelock and ceasing to update20:21
jeblairfungi: i'll do a manual flock+run then20:22
openstackgerritgreghaynes proposed openstack-infra/nodepool: Test different provider and dib image names  https://review.openstack.org/22488620:22
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: Fix broken CI due to discoverd/inspector rename  https://review.openstack.org/22488720:22
fungibut do mean that a full resync won't work without manual assistance20:22
greghaynesjeblair: ^ very trivial way to test ;)20:22
*** mmedvede has quit IRC20:22
fungiwe're around 2/3 complete on the rsyncs now too20:22
fungialso we're good and clear on the old ttl for the pypi mirror in hpcloud now, so i'll approve my wip revert20:23
fungiand i'll also apply it manually because it'll take forever to land without enough nodes to run jobs20:24
*** jsavak has quit IRC20:25
fungijeblair: are you okay with me temporarily disabling puppet on nodepool.o.o while 224767 is working its way through the ci? is that going to further cramp your ovh testing?|20:25
jeblairfungi: no it's fine; i'm stuck until https://review.openstack.org/224821 lands and we restart20:26
*** jsavak has joined #openstack-infra20:26
jeblairfungi: heh20:26
jeblairfungi: it looks like the disable patch hasn't actually landed20:26
jeblairfungi: you could probably abandon both and just run puppet to reset :)20:27
fungiahh, and yes i never enabled it the first time because of that20:27
fungii'll do so20:27
*** lucasagomes has quit IRC20:29
fungiokay, that change and its revert have been abandoned, puppet agent is enabled again on nodepool.o.o and i've manually run puppet agent --test there to get things back on track20:29
jeblairHello Rackspace Customer, Thank you for your payment of $86.90, your credit card on file has been charged based on your invoice dated 17-Sep-15.20:30
funginow to see if i can spot what's going on in rax that has our booting percentage so high20:30
jeblairi wish i had the _foggiest_ idea which account that was about.20:30
*** gyee has joined #openstack-infra20:31
fungijeblair: is that on one of our infra accounts?20:31
jeblairfungi: no idea!20:31
*** julim has quit IRC20:31
fungioh!20:31
greghaynesjeblair: at the bottom of that checkForMissingDiskImage I noticed you didnt change image.name to image.diskimage for the uploadImage call... was that knowingly done?20:31
fungiwow, yeah that's great20:31
fungijeblair: did you maybe have an iopenedthecloud.com promotional 2-year discount and it just ran out this month?20:32
greghaynesasking because im kind of on the fence for how I want to fix this20:32
jeblairgreghaynes: hrm, it looks like uploadImage may need a similar set of fixes.20:34
greghaynesjeblair: yep, thats what im running in to :)20:34
greghaynesjeblair: working on a fix now20:34
jeblairgreghaynes: cool, all yours, thx20:34
fungilots of 500 errors booting in hpcloud. checking for fips, but possible we're overrunning an api throttle with the big spike in boot calls20:35
*** mmedvede has joined #openstack-infra20:35
jeblairfungi: ah, yeah that was it.  deleted :)20:36
*** miqui has quit IRC20:36
*** tiswanso has quit IRC20:37
*** ccarrara has joined #openstack-infra20:38
*** doug-fish has joined #openstack-infra20:38
*** doug-fish has quit IRC20:38
*** doug-fish has joined #openstack-infra20:39
*** tonytan4ever has quit IRC20:39
*** esker has quit IRC20:41
*** Sukhdev has joined #openstack-infra20:42
*** jsavak has quit IRC20:42
*** jsavak has joined #openstack-infra20:43
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add signatire for 1496678  https://review.openstack.org/22442520:43
fungii think it must have been api throttles because i'm not seeing new ones in the log and there weren't really any stale disassociated fips20:44
fungiand i see more and more nodes coming online20:44
*** lucasagomes has joined #openstack-infra20:44
fungialready 18 running jobs in hpcloud20:44
*** glauco has quit IRC20:44
*** bswartz has quit IRC20:45
*** krtaylor has quit IRC20:45
mriedemanyone know what caused this to spike today? http://logs.openstack.org/39/193139/1/gate/gate-grenade-dsvm/0086d24/console.html#_2015-09-17_14_32_11_36820:45
*** baoli has quit IRC20:46
*** baoli has joined #openstack-infra20:46
jeblairmriedem: i see failures to pip install ansible, so probably hpcloud deleting our mirror account.20:46
*** kushal has quit IRC20:47
fungiyep, it subsided when we reconfigured nodepool to stop booting new nodes in hpcloud20:47
fungiin theory20:47
fungithe timing on that log certainly looks like it's from the timeframe where it was still broken anyway20:47
openstackgerritMerged openstack-infra/nodepool: Bump pbr requirements to >=1.3  https://review.openstack.org/22328020:48
mriedemah yeah, ok20:48
mriedemyeah it was definitely just a spike today in logstash20:48
mriedembuild_node probably is all hpcloud too20:48
mriedemyup20:48
fungiclarkb: the latest update on 1348954 is very promising20:48
fungiclarkb: er, on bug 134895420:48
openstackbug 1348954 in python3-defaults (Ubuntu) "update Python3 for trusty" [Undecided,New] https://launchpad.net/bugs/134895420:48
clarkbfungi only a year late20:49
*** sridhar_ram has joined #openstack-infra20:49
*** bltavares has quit IRC20:49
*** e0ne has quit IRC20:50
*** ccarrara has quit IRC20:51
openstackgerritJames E. Blair proposed openstack-infra/puppet-odsreg: Deploy as a wsgi app  https://review.openstack.org/22435320:51
openstackgerritMerged openstack-infra/puppet-etherpad_lite: Ensure abiword package is always present  https://review.openstack.org/22410320:52
fungiwe're now at 165 nodes running jobs in hpcloud, so hopefully we'll start catching up20:52
jeblairfungi: are we sure we want abiword on there?  ^20:52
*** annegentle has quit IRC20:52
jeblair(i read the commit message as "ensure we will need to log in and kill abiword periodically")20:53
pleia2:\20:53
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for ansible not on hpcloud nodes bug 1497006  https://review.openstack.org/22489320:53
openstackbug 1497006 in OpenStack-Gate "gate jobs fail because ansible is not installed on hpcloud nodes" [Undecided,New] https://launchpad.net/bugs/149700620:53
fungijeblair: apparently etherpad is injecting an error about missing abiword into every new pad created20:53
openstackgerritgreghaynes proposed openstack-infra/nodepool: Correct missing DIB image detection  https://review.openstack.org/22482120:53
jeblairfungi: yeah, would be nice to turn that off.20:53
*** aeng has joined #openstack-infra20:54
greghaynesjeblair: ^ think thats it20:54
fungialso, it might make sense to find out if anyone's relying on the pad import/export features20:54
jeblairbut still, i feel like we're going from "spurious error message telling us we are broken when we aren't" to "actually broken but with no error message" :/20:54
fungibut i agree, i'd rather we just disabled that, i simply haven't had an opportunity to dig in and find out whether it's possible20:54
pleia2the error message it gives only tells you how to turn off all errors, with is not optimal20:55
pleia2s/with/which20:55
*** tongli has quit IRC20:55
fungii guess the current situation does technically disable import/export, albeit with some associated ugly20:55
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add odsreg server  https://review.openstack.org/22387120:56
openstackgerritAkash Gangil proposed openstack-infra/project-config: Gerrit bot request for vmware-nsx irc channel  https://review.openstack.org/22489820:57
jeblairfungi, pleia2: can you review/aprv "status:open topic:odsreg" ?  then i can spin up the server for ttx to use tomorrow20:58
*** gangil has joined #openstack-infra20:58
fungijeblair: was just pulling that up since i saw gerritbot chiming one in here20:58
*** yfried has joined #openstack-infra20:58
*** yfried has quit IRC20:59
*** hodos|2 has quit IRC20:59
*** matrohon has quit IRC20:59
jeblairfungi: and i fixed your (0) comment in a followup21:00
*** abregman has quit IRC21:00
*** thorst has quit IRC21:01
gangilHi, I wanted to make sure that I registered openstack-vmware-nsx the right way. I just came across this doc for IRC channels for various openstack projects  https://wiki.openstack.org/wiki/IRC21:02
gangilCould anyone please help me with that?21:02
pleia2jeblair: reviewing now21:03
*** regXboi has quit IRC21:04
*** jsavak has quit IRC21:05
*** david-lyle has joined #openstack-infra21:06
pleia2gangil: you want to add the openstackinfra as a founder, see: http://docs.openstack.org/infra/system-config/irc.html#access21:07
fungijeblair: have you been testing that vhost on trusty or precise?21:07
jeblairfungi: trusty21:07
fungiinteresting21:07
fungii wonder why the permissions in those directory blocks don't need a "satisfy all" directive21:08
jeblairfungi: maybe mod_access_compat ?21:08
gangilpleia2: done21:08
fungier, "satisfy any" i mean21:08
*** bswartz has joined #openstack-infra21:08
fungijeblair: oh! that would be why21:08
jeblairfungi: though i'm currently trying to figure out why that's not sufficient for pypi21:08
*** david-lyle has quit IRC21:09
*** baoli has quit IRC21:09
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: Fix broken CI due to discoverd/inspector rename  https://review.openstack.org/22488721:09
*** baoli has joined #openstack-infra21:09
*** alivigni has quit IRC21:09
*** eharney has quit IRC21:11
*** trown is now known as trown|outttypeww21:11
*** aysyd has quit IRC21:12
*** Sukhdev has quit IRC21:12
*** mdenny has quit IRC21:13
*** baoli has quit IRC21:14
*** ldnunes has quit IRC21:15
*** olaph has joined #openstack-infra21:16
*** mdenny has joined #openstack-infra21:16
fungiokay, the rax pypi mirror rsyncs are completed, so i'm going to swap them in and get full bandersnatch state refreshes going on them now21:16
*** dkranz has quit IRC21:17
*** ajmiller has quit IRC21:19
*** tonytan4ever has joined #openstack-infra21:19
*** tonytan4ever has quit IRC21:20
*** tiswanso has joined #openstack-infra21:22
*** dims_ has quit IRC21:23
*** armax has joined #openstack-infra21:24
*** jamielennox|away is now known as jamielennox21:24
openstackgerritJames E. Blair proposed openstack-infra/puppet-httpd: Add "Satisfy any" to default vhost template  https://review.openstack.org/22490521:26
*** tiswanso has quit IRC21:26
jeblairfungi, mordred: ^ that's needed for pypi mirrors on trusty21:26
*** dprince has quit IRC21:27
fungiyep, that looks like something i would expect21:27
jeblairfungi: i'm guessing that isn't needed in odsreg because "wsgi app" or something?21:28
fungii'm kinda stumped as to why odsreg doesn't need it, unless wsgi is handling the authorization bit, yeah21:29
fungiwsgi is sort of a black box to nme21:29
clarkbjeblair: greghaynes I am not sure https://review.openstack.org/#/c/224821/3/nodepool/nodepool.py is correct, that assumes you will never use the same diskimage as the base for two different provider images21:30
clarkbjeblair: greghaynes maybe they have different flavors or other differences21:30
clarkbI think for upload image you want to apss the provider image then in uploadImage look up what specific diskimage needs to be uploaded for that image21:30
clarkbunfortunately nodepool has no way of sharing images in a provider between flavors right now21:31
jeblairoh, that's my change?21:31
clarkbjeblair: ya but greghaynes updated it to fix the test21:32
clarkbI think the test only works if you don't share a diskimage in multiple provider images21:32
greghaynesclarkb: lets make a test for that21:32
jeblairclarkb: well, he updated it to further correct the bug in uploadImage21:32
jeblairthough, i think that could have been a separate patch :)21:32
clarkbjeblair: right that21:32
waynrzaro: pong21:32
greghaynesjeblair: woops, sorry21:33
greghaynesI was hoping to make a thing with a passing test21:33
clarkblet me leave a comment then it can be cleaned up however21:33
greghaynesclarkb: So I dont see how the behavior would be different now - before it was looking up the image in a spcific provider21:33
*** mriedem has quit IRC21:33
greghaynesclarkb: or maybe I dont understand exactly what youre saying21:33
clarkbgreghaynes: I think the old behavior was wrong too21:34
greghaynesclarkb: oh, well then yes21:34
clarkbgreghaynes: but if we are going to fix it lets actually fix it21:34
greghaynesclarkb: agreed21:34
*** mdenny has quit IRC21:34
jeblairclarkb: thanks.  part of why i don't understand your comment is i don't know whether you are criticizing my change or greghaynes :)21:34
greghaynesclarkb: what is extra :( is that I did rework this to be less crazypants in my nodepool builder patches21:34
clarkbcommented21:35
*** mdenny has joined #openstack-infra21:35
jeblairclarkb, greghaynes: i'm not sure i full understand what greghaynes wrote yet, but based on our earlier conversation, i did expect uploadImage to need an additional argument (or at least know both names)21:36
jeblair(that's tentative agreement with clarkb's comment)21:36
greghaynesclarkb: my thought on fixing that now - I intentionally didnt change the function signature because theres more than one place where we call that and kind of wanted to get it fixed with less chance of breaking more things21:37
openstackgerritMerged openstack-infra/puppet-odsreg: Initial commit  https://review.openstack.org/22387021:37
*** cloudtrainme has quit IRC21:37
*** tonytan4ever has joined #openstack-infra21:37
clarkbgreghaynes: it should only be called in 2 places21:37
fungiokay, i have root-owned screen sessions with full pypi mirror state refreshes going on all three rax mirrors now21:37
greghaynesclarkb: grep counts 621:37
clarkboh maybe tests?21:38
jeblairfungi: similarly i have 2 bootstraps in screen21:38
greghaynesclarkb: not including tests21:38
greghaynesclarkb: oh, one of them is in provider, so 421:38
clarkbgreghaynes: huh anyways the name of the arg is image_name we aren't passing that in that change21:38
clarkbgreghaynes: we need to pass the image name and can lookup the diskimage from there or we need to pass the diskimage too21:38
clarkbbut diskimage alone isn't unique key21:38
fungijeblair: yep, and as discussed i'm basically running the command from the crontab minus wrapping with timeout21:38
greghaynesclarkb: its passing diskimage + provider21:39
clarkbgreghaynes: correct thats not unique21:39
clarkbgreghaynes: you can have multiple provider images that use the same diskimage21:39
fungiso still with flock21:39
fungiand the log redirects21:39
clarkbgreghaynes: it is one way of having multiple flavors in use21:39
greghaynesclarkb: right. So this was just horribly borken21:39
clarkbgreghaynes: yes21:39
jeblairgreghaynes: yes21:39
pleia2jeblair: reviewed and +2ed those changes, but they're still going through checks, can +A at will, I need to go get my cat from the vet :)21:39
greghaynesclarkb: ugh21:39
jeblairit only ever worked because we _happened_ to use the same names21:40
greghaynesheh, yea, saved by the bug21:40
greghaynesok, I can dive in on that21:40
jeblairpleia2: feel free to +A things before check has finished, that's what gate is for21:40
jeblairpleia2: i know they don't like that in openstackland, but we're more relaxed :)21:40
*** andersonvom has joined #openstack-infra21:41
*** amuller has quit IRC21:43
*** abregman has joined #openstack-infra21:43
*** ddieterly has joined #openstack-infra21:43
jeblairgreghaynes, clarkb: yeah, my first mental cut of the fix for this is basically: pass both image names, use dib image name for getOrderedReadyDibImages, and use the provider image name the rest of the time.  more or less.  would expect very little code change.21:44
*** ddieterly has joined #openstack-infra21:44
greghaynesjeblair: yea, using the provider name is better, I was hoping to get away without changing the arg types but didnt realize how broken it is21:45
fungii've also now detached and deleted the old cinder volumes from the rax pypi mirrors, so cleanup is finished21:46
greghaynesactually I did change it anyways, /me make it less bad21:46
*** tlian has quit IRC21:47
*** jaypipes has quit IRC21:47
*** amitgandhinz has quit IRC21:48
*** dims_ has joined #openstack-infra21:49
openstackgerritAkash Gangil proposed openstack-infra/project-config: Gerrit bot request for vmware-nsx irc channel  https://review.openstack.org/22489821:51
*** tiswanso has joined #openstack-infra21:52
fungiokay, popping out to get a bite to eat, will be back in a while to check in on the pypi mirror update progress21:53
openstackgerritgreghaynes proposed openstack-infra/nodepool: Correct missing DIB image detection  https://review.openstack.org/22482121:53
greghaynesclarkb: jeblair ^ is that what you mean?21:53
*** DericHorn-HP has quit IRC21:54
*** r1chardj0n3s has left #openstack-infra21:54
*** bhunter71 has joined #openstack-infra21:54
clarkbyes21:56
jeblairgreghaynes: that looks like it should work21:56
greghaynestrying to fix up the test now...21:56
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Correct exception when waiting for image build  https://review.openstack.org/22482321:57
jeblairrebase ^21:57
*** Sukhdev has joined #openstack-infra21:57
*** gordc has joined #openstack-infra21:58
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Test different provider and dib image names  https://review.openstack.org/22488621:58
jeblairrebased that too21:59
*** sabeen1 has joined #openstack-infra21:59
greghaynesclarkb: jeblair when you do the image-update command - you are specifying the provider image.name, not the diskimage name, correct?22:00
jeblairgreghaynes: not sure i've used it; lemme see22:01
clarkbyes22:02
greghayneshrm, I think there might be a similar break for nodepoolcmd image-update22:03
jeblairgreghaynes: yes, looks like it :)22:03
*** NikitaKonovalov has quit IRC22:03
jeblair(er, unsure about break, i was confirming cmd syntax)22:03
greghaynesyep22:03
*** jd__ has quit IRC22:03
*** fbo has quit IRC22:03
greghaynesIt is really cool that we have tests now, though22:04
*** fbo has joined #openstack-infra22:04
*** jd__ has joined #openstack-infra22:04
*** ddieterly has quit IRC22:04
*** Sukhdev has quit IRC22:05
*** NikitaKonovalov has joined #openstack-infra22:05
*** stevemar has quit IRC22:05
*** tonytan4ever has quit IRC22:05
*** shivharis has joined #openstack-infra22:06
*** annegentle has joined #openstack-infra22:08
*** ozamiatin has joined #openstack-infra22:08
*** bhunter71 has quit IRC22:09
*** dimtruck is now known as zz_dimtruck22:09
ruagairstill looking for the Storyboard dump if any root folk are about and can slot it in :-)22:10
jeblairruagair: oh hi.  i guess i'm it :)22:10
*** achanda has quit IRC22:11
*** doug-fish has quit IRC22:11
*** stevemar has joined #openstack-infra22:11
ruagairThanks jeblair. Greatly appreciated. My phabricator instance looks sad and lonely at present :-D22:12
jeblairso looks like i need to strip these tables: accesstokens, authorizationcodes, refreshtokens22:12
*** bpokorny has quit IRC22:12
*** bpokorny has joined #openstack-infra22:12
*** ozamiatin has quit IRC22:13
*** ZZelle_ has quit IRC22:14
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Support copyartifact build selector param  https://review.openstack.org/13377422:14
*** stevemar has quit IRC22:14
*** hodos has joined #openstack-infra22:16
*** dims_ has quit IRC22:17
*** dims_ has joined #openstack-infra22:20
*** ozamiatin has joined #openstack-infra22:20
*** weshay has quit IRC22:20
*** armax has quit IRC22:21
*** sigmavirus24 is now known as sigmavirus24_awa22:21
jeblairruagair: grab https://storyboard.openstack.org/db-dump/storyboard.sql.gz22:21
jeblairruagair: and let me know when you have it22:21
greghaynesjeblair: clarkb ok, so this kind of breaks our image-update semantics. Multiple providers could have multiple images with name foo which point to different diskimages. In that case we should rebuild all the images being pointed to?22:22
jeblairi've disabled puppet because i believe it will reinstall storyboard which will remove that directory22:22
*** bltavares has joined #openstack-infra22:22
jeblairgreghaynes: image-update takes a provider name (or the special case 'all')22:22
jeblairgreghaynes: so in normal use, we have provider+image22:22
gangilhttps://review.openstack.org/#/c/224898/ reviews would be great :) thanks!22:22
greghaynesjeblair: yes, in the all case though22:22
jeblairgreghaynes: yes, i think so.  that probably breaks the "dib_image_built" part of that doesn't it?22:23
greghaynesyep22:23
greghayneswell, the code doesnt even slightly resemble something that might be able to do multiple builds22:24
jeblairgreghaynes: yeah, that's the only thing that makes sense to me22:24
greghaynesok. I think this is where were heading but to codify it a bit more - nodepool itself should only deal with provider image names, not diskimage names at all22:25
jeblairgreghaynes: yeah; i think what we want (for the all case) is: for each provider image, update its diskimage if it has not already been done, then upload.22:26
jeblairgreghaynes: right22:26
greghaynesThis is important when nodepool builders break out diskimages to a config that nodepool doesnt have22:26
*** achanda has joined #openstack-infra22:26
*** tiswanso has quit IRC22:27
jeblairgreghaynes: (so normally, we would update the ubuntu-trusty diskimage once and then upload that to 8 providers; but if there were 2 diskimages involved, they would both get updated)22:28
*** mriedem has joined #openstack-infra22:28
greghaynesyep22:28
greghaynessounds good22:28
*** annegentle has quit IRC22:30
*** mriedem1 has joined #openstack-infra22:33
jeblairinteresting... slack seems to be receiving this irc channel22:33
greghaynesslack?22:34
jeblairthe private chat tool for teams startup du jour22:34
clarkbits like hipchat but hipper22:34
pleia2hah22:34
jeblairi suspect someone has it piped into a slack channel automatically22:34
* greghaynes looks at mordred22:35
*** mriedem has quit IRC22:35
greghaynesif anyone was to try that kind of thing...22:35
jeblairgreghaynes: mordred seems to be good at using irc to irc :)22:35
greghaynesdoesnt he have some crazy mutiple layers of bouncer setup?22:36
greghaynesmight have crossed the streams22:36
clarkbno thats dansmith and sdague22:36
jeblair!say hello22:36
openstackjeblair: Error: "say" is not a valid command.22:36
jeblairircbot say hello22:36
jeblairnope, not that one ;)22:36
*** annegentle has joined #openstack-infra22:37
*** hdd has quit IRC22:40
*** hdd has joined #openstack-infra22:41
*** hrou has quit IRC22:45
tonybmriedem1: /win 8022:45
tonyb*sigh* mriedem1 Thanks for that change.22:45
*** ddieterly has joined #openstack-infra22:47
openstackgerritDarragh Bailey proposed stackforge/python-jenkins: Provider helper classes for plugin data management  https://review.openstack.org/15431622:47
ruagairabout to be airbourne jeblair. Will check on the other side. Thanks!22:47
*** ddieterly has quit IRC22:47
*** ddieterl_ has joined #openstack-infra22:48
*** erlon has quit IRC22:48
jlvillalgreghaynes: In disk-image-builder is there an easy way to get more verbose logging of what the image is doing when it is booting up?22:48
greghaynesjlvillal: -x22:48
*** ddieterly has joined #openstack-infra22:48
jlvillalSweet :)22:48
greghaynesjlvillal: oh, booting22:48
clarkbthats independent of dib22:49
greghaynesjlvillal: dib doesnt know anything about what goes on at boot - it just makes the image22:49
clarkbdepends on your init system22:49
jlvillalI was doing a: disk-image-create ubuntu baremetal localboot dhcp-all-interfaces -o my-image22:49
jlvillalAnd seeing errors on boot, so was hoping to make it be more verbose.22:49
jlvillalI will keep looking22:50
greghaynesjlvillal: yea, dib really doesnt have anything to do with that, youre gonna have to look at the upstart logs or similar22:50
jlvillalgreghaynes: Yeah it fails to finish the boot :(  So I don't have access to any logs.22:50
*** ozamiatin has quit IRC22:50
greghaynesjlvillal: console log?22:50
clarkbor boot recovery instance22:51
jlvillalgreghaynes: I got that :)22:51
jeblairor worse, try to figure how to access the logs in systemd22:51
greghaynesjeblair: only possible suggestions please22:51
clarkbjeblair journalctl -something -somethingelse22:51
jeblairwhich is going to be really hard if you can't run the journal command thingy22:51
jlvillalI can see tftp failing with 'c' not valid option22:51
greghaynes:)22:51
jlvillalAnd then a curl command fails, but I don't see what the command is22:51
jlvillalI will keep investigating. Thanks22:51
clarkbjeblair wow I never considered that before and just wait for binary format to be versioned22:52
*** ddieterl_ has quit IRC22:52
*** zeih has joined #openstack-infra22:53
*** ddieterly has quit IRC22:53
*** Piet has joined #openstack-infra22:53
*** Piet has quit IRC22:54
*** gildub has joined #openstack-infra22:55
*** ddieterly has joined #openstack-infra22:55
*** gangil1 has joined #openstack-infra22:56
*** andersonvom has quit IRC22:56
*** gangil has quit IRC22:57
*** zeih has quit IRC22:58
*** gordc has quit IRC22:58
*** dims_ has quit IRC22:59
*** austin81 has left #openstack-infra22:59
*** mrmartin has joined #openstack-infra22:59
*** david-lyle has joined #openstack-infra22:59
*** bpokorny has quit IRC23:00
*** bpokorny has joined #openstack-infra23:01
*** dangers is now known as dangers_away23:02
*** krtaylor has joined #openstack-infra23:02
*** sridhar_ram has quit IRC23:03
*** annegentle has quit IRC23:03
*** DericHorn-HP has joined #openstack-infra23:04
*** bltavares has quit IRC23:06
*** bltavares has joined #openstack-infra23:07
*** rfolco has quit IRC23:08
*** bltavares has quit IRC23:11
jeblairi'm deleting leaked hpcloud ports23:14
*** hemna is now known as hemnafk23:14
*** lucasagomes has quit IRC23:16
*** MarkAtwood has quit IRC23:18
*** hodos has quit IRC23:21
*** MarkAtwo_ is now known as MarkAtwood23:21
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Reorder imports to match hacking guidelines  https://review.openstack.org/13346523:25
*** trown|outttypeww is now known as trown23:25
*** baoli has joined #openstack-infra23:25
*** MarkAtwood has quit IRC23:29
tonybis the gerrit UI slow for others?23:29
*** sergmelikyan has joined #openstack-infra23:33
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Support default string values for variables  https://review.openstack.org/18011723:33
*** ozamiatin has joined #openstack-infra23:36
*** markvoelker has quit IRC23:36
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Separate out custom yaml tags into separate classes  https://review.openstack.org/18037023:37
*** ivar-lazzaro has quit IRC23:37
openstackgerritlifeless proposed openstack/requirements: Remove workaround for pypy  https://review.openstack.org/22492423:37
*** ddieterly has joined #openstack-infra23:38
*** ivar-lazzaro has joined #openstack-infra23:38
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Support lazy resolving of include yaml tags  https://review.openstack.org/6358023:39
pleia2happy to report that once this change (documentation) lands, we can mark the Zanata spec as implemented :) https://review.openstack.org/#/c/222314/23:40
pleia2good week \o/23:41
*** dkehn has quit IRC23:43
*** romcheg has quit IRC23:44
*** zns has quit IRC23:45
*** mdenny has quit IRC23:47
*** ddieterl_ has joined #openstack-infra23:48
*** armax has joined #openstack-infra23:48
*** alexpilotti has joined #openstack-infra23:49
ianwhas something changed on etherpad?  when i paste in a bunch of logs, it strips otu all the newlines and really screws it up23:49
geoffarnoldQuestion: how come oslo-related blueprints don't show up in http://status.openstack.org/release/ ? What other projects don't show up there?23:49
*** xyang1 has quit IRC23:49
*** dkehn has joined #openstack-infra23:50
*** sergmelikyan has quit IRC23:50
*** ddieterly has quit IRC23:51
*** DericHorn-HP has quit IRC23:52
jamielennoxcan anyone tell me what this failure is: http://logs.openstack.org/50/217450/1/gate/gate-keystoneclient-dsvm-functional/ea52785/console.html#_2015-09-15_22_50_49_49023:53
jamielennoxit seems all the tests passed and then there's an ansible issue23:53
tonybfungi: https://wiki.openstack.org/wiki/Election_Officiating_Guidelines#Generate_a_list_of_voters  has instructions on how to generate a list of voter email addresses23:53
openstackgerritJohn L. Villalovos proposed openstack/diskimage-builder: Fix variable misspelling error  https://review.openstack.org/22492823:53
tonybfungi: a) is that basically what you'll do? ; b) if not, does what youll do for us need any special permissions?23:54
ianwjamielennox: nah, ansible is perfect, i think the issue is above that http://logs.openstack.org/50/217450/1/gate/gate-keystoneclient-dsvm-functional/ea52785/console.html#_2015-09-15_22_50_45_13623:55
ianwjamielennox: is this a new test or something?23:55
tonybfungi: for example I have all the repos from "ssh review gerrit ls-projects" on my laptop23:55
jamielennoxianw: no, it's an old one, a gate job on keystoneclient stable - so something that doesn't get run often23:56
*** baoli has quit IRC23:56
fungitonyb: that's not at all what we do for official technical elections, because what we do involves getting table dumps from the gerrit database to identify primary contact addresses and change owners (so they can be confirmed to be cla signers)23:56
*** sflanigan has joined #openstack-infra23:57
jamielennoxianw: hmm, weird - it still seems like something that wouldn't be under the control of that test23:57
tonybfungi: Ah okay good to know.23:57
fungitonyb: i think that wiki was something someone came up with as an "almost as good" suggestion for unofficial projects wanting to conduct their own elections23:57
tonybfungi: okay.  I'll edit the wiki to make that clear so you don't get asked again ;P23:58
*** ybathia has quit IRC23:58
*** thorst has joined #openstack-infra23:59
*** zns has joined #openstack-infra23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!