Saturday, 2016-10-01

*** yamamoto has quit IRC00:00
jeblairpabelanger: wow you uploaded that *just* in time00:00
pabelangerwas too busy copying the original commit message00:00
jeblairpabelanger: heh, i think the commit message is wrong, but let's leave it and merge+restart asap00:01
*** M-docaedo_vector has quit IRC00:04
*** thorst has quit IRC00:05
*** dkehn has joined #openstack-infra00:07
jeblairpabelanger: when it lands, i will update all the launchers and hard restart them00:08
openstackgerritMerged openstack-infra/zuul: Strip leading / from afs targets  https://review.openstack.org/38067300:09
mgagneI found some cases where Gerrit replies with the exact reason why the push was refused: https://bugs.chromium.org/p/gerrit/issues/detail?id=1728 No luck with Gerrit 2.11 :-/00:09
mgagnemaybe the "feature" got dropped00:09
*** xarses has joined #openstack-infra00:10
jeblairhard restarting zuul launchers now00:10
*** amitgandhinz has quit IRC00:10
pabelangerack00:11
*** dfflanders has joined #openstack-infra00:11
asselin_mgagne, any idea what I'm doing wrong? This works in pycharm, but not on the command line...?? http://paste.openstack.org/show/583702/00:12
*** kaisers2 has joined #openstack-infra00:12
*** kaisers has quit IRC00:12
mgagneasselin_: what's the value of type_cl ?00:13
*** tonytan4ever has quit IRC00:13
jeblairpabelanger: telnet://149.202.172.231:1988500:13
mgagneasselin: there is an example with import_class here: https://github.com/openstack/nova/blob/master/nova/compute/__init__.py#L4700:13
pabelangerjeblair: watching00:13
mgagneasselin_: you can find example of values in CELL_TYPE_TO_CLS_NAME above00:14
asselin_mgagne, http://paste.openstack.org/show/583703/00:14
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Make bug name, type, and url explicit  https://review.openstack.org/37511300:14
asselin_mgagne, code ^^00:14
jlvillalamrith, On the gist you shared: https://gist.github.com/anonymous/da3bb0fb30a7f47bf1c6e7a7b4b0233900:15
jlvillalamrith, All those *.pyc files could be in the .tox/ directory.00:15
asselin_i'll try other that snippet you provided00:16
jlvillalamrith, We shouldn't try to stop *.pyc files in .tox/00:16
mgagneasselin_: could it be type is a reserved name?00:16
* asselin_ renames 'type'00:16
*** adrian_otto has quit IRC00:16
jlvillal'type' is a reserved name in python.00:17
mgagnejlvillal: not according to https://docs.python.org/3.0/reference/lexical_analysis.html00:17
jlvillalmgagne, Built in function then00:17
mgagnejlvillal: but could be something else already defined00:17
*** salv-orlando has joined #openstack-infra00:17
mgagneyea00:17
jeblairyeah, you can shadow it, but it's frowned upon00:17
*** tiswanso has joined #openstack-infra00:17
mgagneಠ_ಠ00:18
jlvillal+1. Don't overwrite built-in functions :)00:18
jlvillalmakes for confusing code to read00:18
*** Apoorva has quit IRC00:18
mgagnebut type_cl is used in your case so shouldn't that much of a problem for import_class00:19
*** claudiub has quit IRC00:19
jlvillalmgagne, Not my case. I'm not doing it. But yes type_cl should be a fine name for a variable.00:19
jeblairpabelanger: http://files.openstack.org/docs/00:20
jeblairpabelanger: \o/00:20
pabelangerNice, just see that00:20
jeblairpabelanger: also http://files.openstack.org/docs/infra/zuul/  still exists!00:20
jeblairpabelanger: so rsync did not recursively delete all of the docs :)00:21
pabelangerexcellent00:21
*** salv-orlando has quit IRC00:21
asselin_renaming didn't work. what does that error mean? file/class not found, or some other permission issue? not much on google.. "ImportError: Import by filename is not supported"00:22
pabelangerjeblair: is the plan to give is a few days for afsdocs jobs to populate everything before changing DNS?00:22
*** tiswanso has quit IRC00:22
mgagneasselin_: I would print the value of those variables and see what they contain. maybe it's not what you expect00:23
mgagneto reproduce, I checkout your change and run the command found in paste?00:23
jeblairpabelanger: yes; i also think we need to chat with the docs folks about when they're comfortable making the switch -- including giving them a chance to trigger any jobs to fill in any holes, or maybe copy stuff over from the ftp site00:23
asselin_mgagne, yes....you'll need that .conf file, and I think the snippet I provided is sufficient00:24
mgagnelooks like I'm not getting the error. outputting Starting query for bug ...00:26
asselin_huh....something w/ my env then....00:27
mgagneI'm using oslo.utils==3.16.000:27
asselin_works in pycharm...not on command line :/00:27
asselin_same00:27
mgagneI don't use pycharm =)00:27
asselin_I'm in a virtualenv..you?00:27
mgagneran this command:  tox -evenv -- elastic-recheck-graph -c elasticRecheckOO.conf -o web/share/elastic-recheck/data/all.json -v queries/00:27
mgagnePython 2.7.200:28
mgagnesuper old python, I can update and see if it changes anything00:28
mgagnenow it fails with: ImportError: No module named logstash-%Y.%m00:30
mgagneand now with ImportError: Import by filename is not supported.00:31
asselin_i tried your command and I get the same error00:32
mgagneso it looks a look like the value is not the one you expect00:32
asselin_ok I see it printed stuff that's that what I expect...I'll dig into that now00:32
mgagneI'm getting a lot more than the class names you pasted above00:32
asselin_yeah, I think I have a bug in the conf parsing....00:34
mgagneasselin_: I think above call to ConfigParser.ConfigParser is wrong00:35
mgagneasselin_: looks like all the provided values are used as default for all sections00:35
*** TheMetaTroll has joined #openstack-infra00:36
asselin_oh...huh...do you know of any helper utils that provide a default for a section missing in a conf file?00:36
mgagneoff the top of my head, not really. I only know about oslo_config =)00:37
mgagneif you don't/can't use oslo_config, maybe you can see what it's doing to define its defaults00:37
*** sdake has quit IRC00:39
asselin_I'll just write a simple wrapper...silly python api00:39
mgagneasselin_: bug_trackers, can't they be defined in setup.cfg ?00:40
mgagneasselin_: or is the goal to allow someone to override them in their config?00:40
asselin_yeah, in the config.00:41
*** TheMetaTroll has left #openstack-infra00:42
*** Apoorva has joined #openstack-infra00:47
*** lin_yang has quit IRC00:50
*** Apoorva has quit IRC00:51
*** hongbin has joined #openstack-infra00:54
*** mtanino has quit IRC00:54
asselin_mgagne, thanks so much. that was it. Working now :)00:55
mgagneawesome =)00:56
*** baoli has joined #openstack-infra00:57
*** ijw has quit IRC00:59
*** yamamoto has joined #openstack-infra01:01
*** tqtran has quit IRC01:04
*** amitgandhinz has joined #openstack-infra01:06
*** yamamoto has quit IRC01:07
*** tphummel has quit IRC01:08
*** ijw has joined #openstack-infra01:08
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Make bug name, type, and url explicit  https://review.openstack.org/37511301:10
*** lezbar has quit IRC01:13
*** lezbar has joined #openstack-infra01:13
openstackgerritMerged openstack/os-client-config: Fix AttributeError in `get_config`  https://review.openstack.org/37657001:15
*** amitgandhinz has quit IRC01:22
*** baoli has quit IRC01:28
*** mriedem has quit IRC01:29
*** thorst has joined #openstack-infra01:31
*** Goneri has joined #openstack-infra01:32
*** thorst has quit IRC01:32
*** kaisers has joined #openstack-infra01:38
*** kaisers has quit IRC01:42
*** ijw has quit IRC01:45
*** ijw has joined #openstack-infra01:45
*** hieulq_ has joined #openstack-infra01:49
*** priteau has joined #openstack-infra01:52
*** hongbin has quit IRC01:52
*** EricGonczer_ has quit IRC01:54
*** baoli has joined #openstack-infra01:57
*** priteau has quit IRC01:57
*** hieulq_ has quit IRC02:00
*** hieulq_ has joined #openstack-infra02:13
*** amitgandhinz has joined #openstack-infra02:18
*** tiswanso has joined #openstack-infra02:18
*** amitgandhinz has quit IRC02:23
*** tiswanso has quit IRC02:24
*** M-docaedo_vector has joined #openstack-infra02:24
*** lezbar has quit IRC02:27
*** lezbar has joined #openstack-infra02:27
*** john-davidge has joined #openstack-infra02:37
*** rhallisey has quit IRC02:37
*** thorst has joined #openstack-infra02:38
*** john-davidge has quit IRC02:42
*** thorst has quit IRC02:47
*** vinaypotluri has quit IRC02:52
*** tuannguyen has joined #openstack-infra02:59
*** thorst has joined #openstack-infra02:59
*** salv-orlando has joined #openstack-infra03:00
*** thorst has quit IRC03:00
*** Sukhdev has joined #openstack-infra03:04
*** tuannguyen has quit IRC03:04
*** david-lyle has quit IRC03:04
*** salv-orlando has quit IRC03:05
*** Jeffrey4l has joined #openstack-infra03:09
*** amitgandhinz has joined #openstack-infra03:19
*** kaisers has joined #openstack-infra03:26
*** tobiash has quit IRC03:28
*** tobiash has joined #openstack-infra03:31
*** kaisers has quit IRC03:31
*** ijw has quit IRC03:32
*** baoli has quit IRC03:33
*** Qiming_ has joined #openstack-infra03:33
*** Qiming_ has quit IRC03:33
*** thorst has joined #openstack-infra03:35
*** thorst has quit IRC03:39
*** lezbar has quit IRC03:42
*** lezbar has joined #openstack-infra03:42
*** Sukhdev has quit IRC03:43
*** Goneri has quit IRC03:46
*** Sukhdev has joined #openstack-infra03:49
*** SteveJo has joined #openstack-infra03:53
*** amitgandhinz has quit IRC03:53
*** Sukhdev has quit IRC03:57
*** Sukhdev has joined #openstack-infra03:57
*** tqtran has joined #openstack-infra04:01
sc68calclarkb: fungi: looks like https://review.openstack.org/379543 is in some kind of loop - gate-tempest-dsvm-platform-fedora24-nv:04:02
sc68calin progress04:02
sc68calit's been running for almost 10 hours04:03
*** yamamoto has joined #openstack-infra04:03
*** yamamoto has quit IRC04:03
*** tqtran has quit IRC04:06
*** rfolco has joined #openstack-infra04:10
*** rfolco has quit IRC04:11
*** hieulq_ has quit IRC04:12
*** harlowja_at_home has joined #openstack-infra04:15
*** vinaypotluri has joined #openstack-infra04:45
*** uxdanielle has quit IRC04:47
*** amitgandhinz has joined #openstack-infra04:50
*** woodster_ has quit IRC04:50
*** salv-orlando has joined #openstack-infra05:02
*** SteveJo has quit IRC05:03
*** yamamoto has joined #openstack-infra05:04
*** coreyob has quit IRC05:05
*** salv-orlando has quit IRC05:07
*** coreyob has joined #openstack-infra05:07
*** yamamoto has quit IRC05:10
*** yamahata has joined #openstack-infra05:14
*** kaisers has joined #openstack-infra05:15
*** harlowja_at_home has quit IRC05:19
*** kaisers has quit IRC05:20
*** amitgandhinz has quit IRC05:23
*** coreyob has quit IRC05:28
*** coreyob has joined #openstack-infra05:29
*** lezbar has quit IRC05:36
*** lezbar has joined #openstack-infra05:36
*** nadya has joined #openstack-infra05:38
*** thorst has joined #openstack-infra05:41
*** baoli has joined #openstack-infra05:45
*** thorst has quit IRC05:49
*** baoli has quit IRC05:50
*** hieulq_ has joined #openstack-infra05:53
*** hieulq_ has quit IRC05:55
*** salv-orlando has joined #openstack-infra06:03
*** hieulq_ has joined #openstack-infra06:07
*** salv-orlando has quit IRC06:07
*** tobiash has quit IRC06:11
*** yamahata has quit IRC06:15
*** amitgandhinz has joined #openstack-infra06:20
*** flepied has quit IRC06:28
*** thorst has joined #openstack-infra06:47
*** nadya has quit IRC06:48
*** lezbar has quit IRC06:51
*** Sukhdev has quit IRC06:52
*** lezbar has joined #openstack-infra06:52
*** amitgandhinz has quit IRC06:53
*** thorst has quit IRC06:55
*** kaisers has joined #openstack-infra06:58
*** tqtran has joined #openstack-infra07:03
*** slagle has quit IRC07:07
*** tqtran has quit IRC07:07
*** cschwede has quit IRC07:07
*** psanchez has quit IRC07:08
*** stevebaker has quit IRC07:09
*** coolsvap has quit IRC07:22
*** hieulq_ has quit IRC07:33
*** salv-orlando has joined #openstack-infra07:33
*** salv-orlando has quit IRC07:34
*** esikachev has joined #openstack-infra07:34
*** salv-orlando has joined #openstack-infra07:34
*** salv-orl_ has joined #openstack-infra07:44
*** salv-orlando has quit IRC07:47
*** mdrabe has quit IRC07:48
*** amitgandhinz has joined #openstack-infra07:50
*** thorst has joined #openstack-infra07:51
*** esikachev has quit IRC07:52
AJaegerclarkb, pabelanger : The index files come from openstack-manuals - so once that one publishes, you should have them...07:52
*** vinaypotluri has quit IRC07:52
AJaegerjeblair, clarkb, pabelanger: So, looking at http://files.openstack.org/docs/ , it looks like openstack-manuals it not pushing content out properly. Translations are available but no content07:54
AJaegerseems that the pushing of content does not work - with all the index files etc.07:57
*** baoli has joined #openstack-infra07:57
* AJaeger reads more backscroll now07:58
*** thorst has quit IRC07:59
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-infra08:01
*** baoli has quit IRC08:01
*** salv-orl_ has quit IRC08:06
* AJaeger just checked jjb, it looks all fine08:07
*** lezbar has quit IRC08:10
AJaegerand yes: openstack-manuals publishes the index files with running "tox -e publishdocs" - openstack-manuals-tox-doc-publishdocs is the job for it.08:10
*** lezbar has joined #openstack-infra08:10
*** nickola has joined #openstack-infra08:13
AJaegerSuccess, there's an index page at http://files.openstack.org/docs/08:17
AJaegerwe previously had a redirect for index page at http://files.openstack.org/docs/developer/ - let me add an index page there now...08:18
AJaegerWait, moving it might be tricky. Can we keep .htaccess on top level?08:21
*** salv-orlando has joined #openstack-infra08:21
*** amitgandhinz has quit IRC08:25
*** lezbar has quit IRC08:27
*** lezbar has joined #openstack-infra08:28
AJaegerOops, the next publish to AFS deletes the content from publishdocs again ;(08:31
AJaegerjeblair: that's something for testing: run openstack-manuals-tox-doc-publishdocs and then openstack-manuals-releasenotes or openstack-manuals-releasenotes. Please double check that the folders contain the markers properly for all jobs.08:33
AJaegerback later...08:34
*** esikachev has joined #openstack-infra08:38
*** esikachev has quit IRC08:43
*** salv-orlando has quit IRC08:52
*** thorst has joined #openstack-infra08:56
*** esikachev has joined #openstack-infra08:59
*** esikachev has quit IRC09:03
*** thorst has quit IRC09:04
*** drifterza has joined #openstack-infra09:11
*** apetrich has quit IRC09:11
*** apetrich has joined #openstack-infra09:14
*** vsaienko has joined #openstack-infra09:15
*** pilgrimstack has joined #openstack-infra09:18
*** esikachev has joined #openstack-infra09:19
*** amitgandhinz has joined #openstack-infra09:22
*** vsaienko has quit IRC09:22
*** vsaienko has joined #openstack-infra09:22
*** esikachev has quit IRC09:24
*** salv-orlando has joined #openstack-infra09:26
*** ramishra has quit IRC09:38
*** ramishra has joined #openstack-infra09:40
*** hashar has joined #openstack-infra09:41
*** amitgandhinz has quit IRC09:41
*** salv-orlando has quit IRC09:56
*** thorst has joined #openstack-infra10:01
*** vsaienko has quit IRC10:05
*** kaisers has quit IRC10:05
*** hashar is now known as hasharAway10:05
*** pilgrimstack has quit IRC10:06
*** thorst has quit IRC10:10
*** pilgrimstack has joined #openstack-infra10:20
*** hieulq_ has joined #openstack-infra10:24
*** hieulq_ has quit IRC10:26
*** pilgrimstack has quit IRC10:27
*** claudiub has joined #openstack-infra10:35
*** salv-orlando has joined #openstack-infra10:37
*** nadya has joined #openstack-infra10:39
*** admcleod_ has joined #openstack-infra10:39
*** admcleod has quit IRC10:40
*** salv-orlando has quit IRC10:46
*** xarses_ has joined #openstack-infra10:46
*** xarses has quit IRC10:50
*** rhallisey has joined #openstack-infra10:56
*** priteau has joined #openstack-infra11:02
*** priteau has quit IRC11:02
*** pilgrimstack has joined #openstack-infra11:02
*** tqtran has joined #openstack-infra11:04
*** pilgrimstack has quit IRC11:06
*** thorst has joined #openstack-infra11:07
*** tqtran has quit IRC11:09
*** nadya has quit IRC11:12
*** thorst has quit IRC11:14
*** nadya has joined #openstack-infra11:36
*** amitgandhinz has joined #openstack-infra11:38
*** dfflanders has quit IRC11:41
*** salv-orlando has joined #openstack-infra11:42
*** nadya has quit IRC11:43
*** salv-orlando has quit IRC11:47
*** HeOS has quit IRC11:48
*** kaisers has joined #openstack-infra11:49
*** nadya has joined #openstack-infra11:50
*** kaisers has quit IRC11:53
*** tobiash has joined #openstack-infra11:55
*** lezbar has quit IRC11:59
*** lezbar has joined #openstack-infra12:00
*** pilgrimstack has joined #openstack-infra12:00
*** drifterza has quit IRC12:00
*** amitgandhinz has quit IRC12:03
*** nickola has quit IRC12:05
*** Krenair has quit IRC12:07
*** flepied has joined #openstack-infra12:09
*** thorst has joined #openstack-infra12:11
*** andreww has joined #openstack-infra12:14
*** lezbar has quit IRC12:16
*** lezbar has joined #openstack-infra12:17
*** xarses_ has quit IRC12:17
*** Krenair has joined #openstack-infra12:18
*** thorst has quit IRC12:19
*** nadya has quit IRC12:27
*** tuannguyen has joined #openstack-infra12:28
*** Naeil has joined #openstack-infra12:28
*** flepied has quit IRC12:29
*** esikachev has joined #openstack-infra12:31
*** lezbar has quit IRC12:32
*** pilgrimstack has quit IRC12:33
*** lezbar has joined #openstack-infra12:33
pabelangerYa, looks like something is deleting the contents12:36
pabelangerI'll check the logs in a bit, have to run some errands this morning12:36
*** esikachev has quit IRC12:37
*** esikachev has joined #openstack-infra12:39
*** tuannguyen has quit IRC12:46
*** EricGonczer_ has joined #openstack-infra12:52
*** david-lyle has joined #openstack-infra12:56
*** oanson has joined #openstack-infra12:57
fungiAJaeger: we should extract out .htaccess content into the apache configuration for the corresponding vhost12:58
*** amitgandhinz has joined #openstack-infra12:59
*** amitgandhinz has quit IRC13:02
*** amitgandhinz has joined #openstack-infra13:02
*** yamamoto has joined #openstack-infra13:03
*** thorst has joined #openstack-infra13:17
*** esikachev has quit IRC13:17
*** tuannguyen has joined #openstack-infra13:21
*** baoli has joined #openstack-infra13:24
*** thorst has quit IRC13:25
*** tuannguyen has quit IRC13:26
*** tuannguyen has joined #openstack-infra13:26
*** tuannguyen has quit IRC13:28
*** EricGonczer_ has quit IRC13:28
*** tuannguyen has joined #openstack-infra13:28
*** baoli has quit IRC13:28
*** tuannguyen has quit IRC13:32
*** tuannguyen has joined #openstack-infra13:32
*** tuannguyen has quit IRC13:33
*** amitgandhinz has quit IRC13:33
*** tuannguyen has joined #openstack-infra13:34
*** yee3794 has quit IRC13:37
*** yee379 has joined #openstack-infra13:37
kfox1111question. is there a document somewhere that describes the different node types that are in the pool?13:38
kfox1111I'm developing a gate job that has k8s in it, and I'm seeing different results depending on where it lands.13:38
*** kaisers has joined #openstack-infra13:38
kfox1111some vm's have multiple nics, some have just one, etc.13:38
*** tuannguyen has quit IRC13:38
pabelangerDon't think we have that documented, but yes I wouldn't expect cloud providers to have a 2nd NIC13:39
pabelangeryou should likely update your jobs to add them, if needed13:39
kfox1111no, I'm saying I'm seeing en example of that. http://logs.openstack.org/56/378056/82/experimental/gate-kolla-kubernetes-deploy-centos-binary-nv/9697408/console.html13:40
kfox1111has an eth0 and and eth1.13:40
kfox1111eth0 146.20.110.66/24 eth1 10.210.33.6/1913:41
pabelangerYes, rackspace has 2 NICs13:41
kfox1111very strange.13:41
kfox1111ah.13:41
pabelangerbut, infracloud-vanilla will be one13:41
*** tiswanso has joined #openstack-infra13:42
kfox1111the other thing that may be causing problems ip space. is there a private address space that wont be used, like racspace there seems to be using a lot of the 10 space.13:42
kfox1111is there a chunk of ip space that's free to use, or is there a chunk thats free per cloud provider and a way to deterimine that?13:42
*** tiswanso has quit IRC13:42
pabelangerI think fungi or clarkb will know more about that, I believe we talked about that topic this week. But didn't see what the answer was13:43
kfox1111k. thanks. :)13:43
*** kaisers has quit IRC13:43
*** baoli has joined #openstack-infra13:43
*** salv-orlando has joined #openstack-infra13:43
*** tiswanso has joined #openstack-infra13:43
*** salv-orlando has quit IRC13:48
clarkbwe dont have any clouds currently using 172.16/1213:49
clarkb(this is where we put devstack floating IPs)13:49
*** yamamoto has quit IRC13:50
clarkbgenerally though we should probably have a "pick a free /20" utility that examines routes on the host to find an availble rfc1918 /2013:50
kfox1111clarkb: yeah, that would be good.13:51
kfox1111ok. I'll use the 172.16/12 space for now then.13:52
kfox1111thanks. :)13:52
*** yamamoto has joined #openstack-infra13:53
fungibut yes, making jobs use some relatively small rfc 1918 network (or series of contiguous ones) with a configurable prefix will make it easier for us to relocate, similar to my suggestions on bug 162913313:55
openstackbug 1629133 in devstack "New neutron subnet pool support breaks multinode testing." [Critical,In progress] https://launchpad.net/bugs/1629133 - Assigned to Dr. Jens Rosenboom (j-rosenboom-j)13:55
kfox1111ah. yeah.13:56
fungisince long term, we can't guarantee that there will be a single safe network across all the various providers who donate resources to us13:57
*** mtanino has joined #openstack-infra13:57
fungiso we may eventually need to make it relocatable at job runtime based on the environment where a particular job lands13:57
*** yamahata has joined #openstack-infra13:58
* kfox1111 nods13:58
*** tphummel has joined #openstack-infra14:08
*** yamahata has quit IRC14:12
*** tqtran has joined #openstack-infra14:19
*** fguillot has joined #openstack-infra14:20
*** thorst has joined #openstack-infra14:21
*** tqtran has quit IRC14:23
*** yamamoto has quit IRC14:26
*** lezbar has quit IRC14:26
*** HeOS has joined #openstack-infra14:26
*** lezbar has joined #openstack-infra14:27
pabelangerYay, 1h5mins since we last had a launch failure, across all our clouds.14:28
pabelangermind you, it is pretty quiet right now14:28
*** thorst has quit IRC14:29
*** amitgandhinz has joined #openstack-infra14:30
*** fguillot has quit IRC14:32
*** dgonzalez has quit IRC14:35
*** dgonzalez has joined #openstack-infra14:35
jeblairAJaeger, pabelanger, fungi: afs is world readable, so you can examine the marker files in there14:37
*** yamamoto has joined #openstack-infra14:40
*** yamamoto has quit IRC14:45
mordredpabelanger: the outbound networking issue in osic ... do we need any additional data collection on it?14:47
*** yamamoto has joined #openstack-infra14:50
clarkbmordred: I don't think so. Cause is bad routes created by devstack/neutron14:52
openstackgerritMerged openstack-infra/gear: Remove discover from test-requirements  https://review.openstack.org/34579514:53
openstackgerritMerged openstack-infra/zuul: Remove discover from test-requirements  https://review.openstack.org/34581114:54
mordredclarkb: oh - you're right. I thought I was seeing the error before devstack ran ... but I was wrong14:55
*** thorst has joined #openstack-infra14:58
*** thorst has quit IRC15:00
*** gouthamr has joined #openstack-infra15:00
AJaegerfungi, so .htaccess should go to system-config?15:00
*** gouthamr_ has joined #openstack-infra15:01
AJaegerjeblair: the openstack-manuals content gets deleted by the next run. So, I wonder whether the marker files are wrong for that case15:01
*** lezbar has quit IRC15:03
*** gouthamr_ has quit IRC15:03
*** drifterza has joined #openstack-infra15:03
*** lezbar has joined #openstack-infra15:03
*** r-mibu has quit IRC15:03
*** r-mibu has joined #openstack-infra15:04
*** amitgandhinz has quit IRC15:04
*** kgiusti has left #openstack-infra15:04
*** gouthamr has quit IRC15:05
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Allow any logdest  https://review.openstack.org/37176415:11
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Simplify puppet cmd generation  https://review.openstack.org/37177815:11
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Add debug argument  https://review.openstack.org/37181115:11
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Don't set detailed-exitcodes when noop  https://review.openstack.org/37182715:11
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Remove basemodulepath from environment config  https://review.openstack.org/37181215:11
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Pass environment to puppet module  https://review.openstack.org/37171015:11
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Add noop argument  https://review.openstack.org/37177415:11
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Make sure environment winds up in basemodulepath  https://review.openstack.org/37167915:11
openstackgerritMonty Taylor proposed openstack-infra/ansible-role-puppet: Sync puppet module changes from upstream ansible  https://review.openstack.org/37183515:11
mordrednibalizer: ^^ I believe I have fixed your comments on the first patch in that series15:11
*** drifterza has quit IRC15:15
jrollclarkb: mordred: fwiw, this appears to fix the osic outbound networking thing for me: https://review.openstack.org/#/c/380023/15:16
jrolle.g. this job would fail here previously http://logs.openstack.org/51/380351/1/check/gate-tempest-dsvm-ironic-ipa-partition-agent_ssh-coreos-src-nv/cf2b637/logs/devstacklog.txt.gz#_2016-09-30_14_56_51_91315:17
*** tiswanso has quit IRC15:18
*** tiswanso has joined #openstack-infra15:19
*** lezbar has quit IRC15:20
*** lezbar has joined #openstack-infra15:20
*** yamamoto has quit IRC15:22
mordredjroll: neat! I'll add a depends-on and see how it goes15:22
*** baoli has quit IRC15:22
openstackgerritMonty Taylor proposed openstack-infra/shade: Add simple field for disabled flavors  https://review.openstack.org/37704915:23
jrollmordred: ++15:24
*** gouthamr has joined #openstack-infra15:24
*** baoli has joined #openstack-infra15:27
*** baoli has quit IRC15:33
*** ijw has joined #openstack-infra15:34
*** tiswanso has quit IRC15:35
fungiAJaeger: sort of. it will be both safer and more efficient if the entries from the .htaccess file are translated into normal apache configuration and added to the vhost template for that site. i've not been following along closely so don't know if provisions have been made for .htaccess support in site content, but i'm assuming (for the reasons i mentioned) that we weren't planning to support15:37
fungiconfiguration in-tree via .htaccess files15:37
*** ijw has quit IRC15:38
mordredfungi: ++15:39
*** yamamoto has joined #openstack-infra15:43
*** yamamoto has quit IRC15:43
fungiit was necessary in cloudsites because of the lack of direct configurability of the web server there15:47
fungibut we have much more flexibility with the new design15:47
*** yamamoto has joined #openstack-infra15:48
*** thorst has joined #openstack-infra15:51
*** HeOS has quit IRC15:56
*** yamamoto has quit IRC16:01
*** amitgandhinz has joined #openstack-infra16:01
*** yamamoto has joined #openstack-infra16:02
*** yamamoto has quit IRC16:02
*** dgonzalez has quit IRC16:08
*** zz_dimtruck has joined #openstack-infra16:13
*** dgonzalez has joined #openstack-infra16:13
*** zz_dimtruck is now known as dimtruck16:13
*** thorst has quit IRC16:17
*** thorst has joined #openstack-infra16:18
*** thorst has quit IRC16:26
*** e0ne has joined #openstack-infra16:29
*** e0ne has quit IRC16:31
*** amitgandhinz has quit IRC16:34
*** pcaruana has quit IRC16:42
kfox1111is there any way to do an interactive job?16:54
*** flepied has joined #openstack-infra16:54
kfox1111I've spent 3 days tryingto get this job to work, and each time it takes 6-10 minutes  to die, then tweak and try again. :/16:55
*** mriedem has joined #openstack-infra16:57
*** EricGonczer_ has joined #openstack-infra16:58
AJaegerfungi, that means less flexibility for the docs team - right now we can merge those on our own. The file http://git.openstack.org/cgit/openstack/openstack-manuals/tree/www/static/.htaccess currently has nearly 500 lines ;(17:00
*** bastelfreak has quit IRC17:01
AJaegerLet's try to make that one as small as possible since we have a new site...17:01
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375017:02
clarkbkfox1111: run it locally amd iterate on that17:04
clarkbfor devstack-gate jobs we write out a reproduce.sh script to help with this17:05
*** kaisers has joined #openstack-infra17:06
fungiAJaeger: yep, i bet we could come up with some sort of hybrid solution (puppet manifest sucks in an apache config chunk from a docs repo and splats it into the middle of the vhost config maybe?)17:20
*** yamamoto has joined #openstack-infra17:20
*** bhavik1 has joined #openstack-infra17:21
*** thorst has joined #openstack-infra17:24
*** Naeil has quit IRC17:24
*** yamamoto has quit IRC17:25
*** amitgandhinz has joined #openstack-infra17:31
*** thorst has quit IRC17:32
*** larainema has quit IRC17:35
*** timello has quit IRC17:38
*** timello has joined #openstack-infra17:43
*** jozwior has joined #openstack-infra17:44
*** mdrabe has joined #openstack-infra17:46
*** dgonzalez has quit IRC17:47
*** dgonzalez has joined #openstack-infra17:47
*** amitgandhinz has quit IRC17:47
*** EricGonczer_ has quit IRC17:49
*** EricGonczer_ has joined #openstack-infra17:49
*** EricGonczer_ has quit IRC17:51
*** jozwior is now known as jozwior_17:58
*** mdrabe has quit IRC17:59
*** mriedem has quit IRC18:00
*** lezbar has quit IRC18:04
kfox1111clarkb: the issue is I keep hitting issues with how openstack's infra is setup, not how my machine is setup.18:04
*** lezbar has joined #openstack-infra18:04
*** vsaienko has joined #openstack-infra18:04
kfox1111k8s uses a bunch of sdnish things and I keep running into things like, an unexpected openstack-infra firewall inside the vm's, different address space on different nodes, etc.18:06
kfox1111different numbers of nics at times.18:06
clarkbyou can also build one of our images and boot it locally18:07
*** jozwior_ has quit IRC18:08
clarkbthere is a script in project-config/tools/build-image.sh18:08
kfox1111ah. that might help a bit. thanks. :)18:08
clarkbto login to that you'll need a config drive with ssh key info whoch will setup keys for root ot just edit the image to add in keys you want18:10
kfox1111hmm... k.18:10
*** Goneri has joined #openstack-infra18:14
clarkbkfox1111: it might also help if you look at how other jobs setup neutron since it does all the sdn things too18:16
clarkbfloating IPs come out of 172.24.4.0/23 and private addresses live in 10.1.0.0/20 iirc. And the firewall isnt an issue because thr firewall allows all communication from or to localhost iirc18:18
*** tuannguyen has joined #openstack-infra18:18
clarkband then neutron manages its owm rules for communcation between VMs18:18
kfox1111clarkb: it does, as the k8s sdn uses the local host and l3 to do everything, rather then l2 like neutron does.18:19
clarkbneutron does l3 too it just knows to edit the rules for itself I think18:19
kfox1111I had to rewrite the openstack iptables rules slightly to get it to work at all:18:20
kfox1111https://review.openstack.org/#/c/378056/92/tools/setup_gate.sh18:20
*** tqtran has joined #openstack-infra18:20
kfox1111line 81+18:20
kfox1111the openstack rules don't have any interface specified, so it ends up getting applied to docker0, flannel, and other internal interfaces.18:21
kfox1111I had to rewrite the openstack rules to apply them only to the physical interfaces.18:21
kfox1111the first attempt at it rewrote it assuming only one external interface... took me a while to figure out there were vm's with 2. :/18:22
clarkbwhy not just allow the traffic you need?18:22
*** lezbar has quit IRC18:23
clarkbthat will probably end up being much simpler18:23
kfox1111its all internal and sdn driven.18:23
kfox1111I just want all internal traffic to be allowed.18:23
kfox1111the problem is, as its sdns, I don't know all the interface names up front. :/18:23
clarkbya we already have a rule for that iirc18:23
*** lezbar has joined #openstack-infra18:23
clarkbbut I guess assumes one end is localhost18:23
kfox1111yeah. theres a lot of cross container traffic.18:24
clarkbya your sdn should manage those rules18:24
clarkbat least thats how neutron seems to have solved the problem18:24
kfox1111it assumes the host firewall won't mess with it. :/18:24
clarkbso that you can talk VM to VM18:24
*** tqtran has quit IRC18:24
kfox1111anyway. I think I finally worked around that. now trying to get the ip space used for the containers to not conflict with the versions in the varous clouds.18:25
kfox1111I think I'm almost there too... just hitting issue after issue after issue. :/18:25
kfox1111and.. failed again.18:26
kfox1111I'm on ps # 92 now. :/18:26
kfox1111seems like its mostly working when it has only one nic.. something about the rackspace vm's make it fail. :/18:28
clarkbfwiw our firewall rules are based on red hat's default with a couple extra rules for snmp and such. So this isnt any thing crazy18:28
*** thorst has joined #openstack-infra18:29
kfox1111yeah.18:29
kfox1111it just was really unexpeted to me to see them. I was expecting security groups, not iptables. :/18:30
kfox1111so I fought for like 8 hours trying to figure it out, before finally realizing why. fealt really dumb. :/18:30
clarkbit turns out not all clouds use security groups so we dont use them18:30
clarkbwee optional cloud features18:30
kfox1111yeah. thats another sore point. :/18:30
kfox1111:)18:30
kfox1111as a devloper of software for clouds, I think that's horible. :/18:31
kfox1111gota go for a while. bbl.18:31
kfox1111thanks for the help.18:31
*** mriedem has joined #openstack-infra18:32
*** baoli has joined #openstack-infra18:35
*** thorst has quit IRC18:35
*** vsaienko has quit IRC18:40
openstackgerritThomas Goirand proposed openstack-infra/project-config: Add deb-glare  https://review.openstack.org/38077918:41
*** amitgandhinz has joined #openstack-infra18:44
*** timothyb89 has quit IRC18:46
*** timothyb89 has joined #openstack-infra18:48
*** vsaienko has joined #openstack-infra18:48
*** gema has quit IRC18:49
*** amitgandhinz has quit IRC18:49
*** baoli has quit IRC18:49
pabelangerTrying to reproduce some rax-iad failures today with centos-7, haven't been able to do so yet.18:51
pabelangerSince we don't have ipv6 working on centos-7, wonder if something around ipv4 is failing18:52
pabelangerI should fix that this week18:52
pabelangerthen we could kick rax-iad to ipv618:52
*** timothyb89 has quit IRC18:53
*** timothyb89 has joined #openstack-infra18:54
pabelangerOtherwise, our last failure on another cloud was 8hours ago:18:54
openstackgerritThomas Goirand proposed openstack-infra/project-config: Retire some wrong deb-os-* and deb-oslo*  https://review.openstack.org/38078318:54
pabelanger2016-10-01 12:03:09,440 ERROR nodepool.NodeLauncher: Timeout launching node id: 4687037 in provider: rax-ord error: Timeout waiting for ssh access18:54
pabelangerthat must be a record :)18:55
*** gema has joined #openstack-infra18:55
clarkbkfox1111: do the logs have more info than "error syncing pod" somehwere?18:56
*** tuannguyen has quit IRC19:09
pabelangerclarkb: I think we need to clean up our DIB cache again, were back up to 8+ hours to build images19:11
pabelanger00:08:37:18 currently19:12
pabelangerI think there was something about our git repos growing over time?19:12
clarkbpabelanger: greghaynes was saying he thought a git repack in thr cacge would help19:12
clarkbso maybe we should try that19:12
pabelangersure19:12
*** winggundamth has quit IRC19:13
*** winggundamth has joined #openstack-infra19:15
greghaynesclarkb: pabelanger Yea, we ran in to a deal downstream where clones were slow and the cache was huge. I just did for dir in cache git repack and it helped19:17
kfox1111clarkb: full logs here: http://logs.openstack.org/56/378056/92/experimental/gate-kolla-kubernetes-deploy-ubuntu-source-nv/8bba313/19:18
kfox1111logs/pods have the individual logs from all the pods present at the time of exit.19:18
greghaynesIt might be worth adding a deal to dib where source-repositories automatically runs a repack after a clone *shrug*19:18
pabelangerYa, like the idea of automatically repack, but not sure how much time that adds19:19
pabelangerI'll see about testing it, maybe write something in crontab19:19
*** rfolco has joined #openstack-infra19:29
*** thorst has joined #openstack-infra19:33
*** oanson has quit IRC19:35
*** vsaienko has quit IRC19:39
*** thorst has quit IRC19:41
*** amitgandhinz has joined #openstack-infra19:45
*** tqtran has joined #openstack-infra19:51
*** amitgandhinz has quit IRC19:51
*** EricGonczer_ has joined #openstack-infra19:52
*** tqtran has quit IRC19:56
*** EricGonczer_ has quit IRC20:01
*** EricGonczer_ has joined #openstack-infra20:06
*** matrohon has joined #openstack-infra20:08
*** EricGonczer_ has quit IRC20:10
*** esikachev has joined #openstack-infra20:16
*** hasharAway has quit IRC20:20
*** rfolco has quit IRC20:25
*** esikachev has quit IRC20:26
*** matrohon has quit IRC20:26
*** Jeffrey4l_ has joined #openstack-infra20:34
*** Jeffrey4l has quit IRC20:35
*** kaisers has quit IRC20:39
*** thorst has joined #openstack-infra20:39
*** kaisers has joined #openstack-infra20:40
*** bhavik1 has quit IRC20:44
*** kaisers has quit IRC20:45
*** annegentle has joined #openstack-infra20:46
*** thorst has quit IRC20:47
*** amitgandhinz has joined #openstack-infra20:48
*** annegentle has quit IRC20:51
*** amitgandhinz has quit IRC20:52
kfox1111arg... how unsettling... everything just started working, and I didn't really change anything. :/20:55
*** esikachev has joined #openstack-infra21:03
*** esikachev has quit IRC21:07
*** lezbar has quit IRC21:09
*** lezbar has joined #openstack-infra21:09
*** esikachev has joined #openstack-infra21:17
*** uxdanielle has joined #openstack-infra21:20
*** baoli has joined #openstack-infra21:23
*** superdan is now known as dansmith21:26
*** Vivek__ is now known as Vivek21:32
*** Vivek has quit IRC21:32
*** Vivek has joined #openstack-infra21:32
*** thorst has joined #openstack-infra21:43
*** amitgandhinz has joined #openstack-infra21:49
*** thorst has quit IRC21:51
*** baoli has quit IRC21:51
openstackgerritKevin Fox proposed openstack-infra/project-config: Kolla Kubernetes tests  https://review.openstack.org/38079521:55
*** thorst has joined #openstack-infra22:00
*** thorst has quit IRC22:00
*** HeOS has joined #openstack-infra22:03
*** rhallisey has quit IRC22:04
*** woodster_ has joined #openstack-infra22:17
*** esikachev has quit IRC22:20
*** rhallisey has joined #openstack-infra22:22
*** amitgandhinz has quit IRC22:23
*** rhallisey has quit IRC22:26
*** rhallisey has joined #openstack-infra22:26
*** lezbar has quit IRC22:28
*** lezbar has joined #openstack-infra22:29
*** apetrich has quit IRC22:31
*** apetrich has joined #openstack-infra22:31
fungikfox1111: heisenbugs are maddening, yes22:33
*** EricGonczer_ has joined #openstack-infra22:33
*** sdake has joined #openstack-infra22:35
*** sdake_ has joined #openstack-infra22:38
*** sdake has quit IRC22:40
*** uxdanielle has quit IRC22:41
*** sdake_ has quit IRC22:44
kfox1111+122:45
kfox1111fungi: what do you think of: https://review.openstack.org/38079522:46
*** rhallisey has quit IRC22:51
fungikfox1111: lgtm. note that you should keep an eye on job name length when invoking tox though, since the shebang munging can end up exceeding the max shebang length for parser invocation22:52
*** rhallisey has joined #openstack-infra22:52
fungiit ends up being something like #!/home/jenkins/workspace/<jobname>/.tox/<envname>/bin/python which, if it exceeds 128 characters (i think that's the total) will cause a very vague error about the shell being unable to find the parser22:53
fungiif you start running into issues, you could drop the gate- prefix from your template names to save 5 characters22:54
*** sdake has joined #openstack-infra22:55
*** asselin__ has joined #openstack-infra22:59
*** rhallisey has quit IRC23:00
*** thorst has joined #openstack-infra23:01
kfox1111fungi: yeah. most of the options will be to the shell script setting up the gate environment. I'm going to switch the tox stuff to just test the deployed openstack.23:02
*** asselin_ has quit IRC23:03
*** yee3795 has joined #openstack-infra23:04
*** mtanino has quit IRC23:04
*** yee379 has quit IRC23:05
*** rhallisey has joined #openstack-infra23:06
*** thorst has quit IRC23:10
*** amitgandhinz has joined #openstack-infra23:21
*** kaisers1 has quit IRC23:30
*** kaisers has joined #openstack-infra23:30
*** kaisers2 has quit IRC23:30
*** kaisers1 has joined #openstack-infra23:31
*** asselin__ has quit IRC23:32
*** asselin_ has joined #openstack-infra23:33
*** asselin__ has joined #openstack-infra23:34
*** asselin_ has quit IRC23:38
*** asselin__ has quit IRC23:39
*** asselin_ has joined #openstack-infra23:39
*** sdake has quit IRC23:41
*** tqtran has joined #openstack-infra23:52
*** amitgandhinz has quit IRC23:53
*** tqtran has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!