*** dmsimard is now known as dmsimard_away | 00:01 | |
*** markvoelker has quit IRC | 00:17 | |
*** j05h has quit IRC | 00:20 | |
*** mdorman has quit IRC | 00:29 | |
*** j05h has joined #openstack-operators | 00:34 | |
*** SimonChung1 has joined #openstack-operators | 00:34 | |
*** SimonChung has quit IRC | 00:34 | |
*** Piet has quit IRC | 00:48 | |
*** signed8bit is now known as signed8bit_ZZZzz | 00:49 | |
*** markvoelker has joined #openstack-operators | 00:52 | |
*** markvoelker has quit IRC | 00:56 | |
*** markvoelker_ has joined #openstack-operators | 00:56 | |
*** j05h has quit IRC | 01:10 | |
*** VW_ has quit IRC | 01:15 | |
*** RaginBajin has joined #openstack-operators | 01:16 | |
*** blairo has joined #openstack-operators | 01:19 | |
*** blair has quit IRC | 01:22 | |
*** blairo has quit IRC | 01:23 | |
*** david-lyle has quit IRC | 01:26 | |
*** Piet has joined #openstack-operators | 01:27 | |
*** RaginBajin has quit IRC | 01:34 | |
*** raginbaj- has joined #openstack-operators | 01:34 | |
*** raginbaj- is now known as raginbajin | 01:34 | |
*** SimonChung has joined #openstack-operators | 01:36 | |
*** SimonChung1 has quit IRC | 01:36 | |
*** klindgren_ has joined #openstack-operators | 01:40 | |
*** blair has joined #openstack-operators | 01:42 | |
*** klindgren has quit IRC | 01:44 | |
*** SimonChung has quit IRC | 01:49 | |
*** raginbajin has quit IRC | 01:51 | |
*** raginbajin has joined #openstack-operators | 01:52 | |
*** Marga_ has quit IRC | 01:54 | |
*** Marga_ has joined #openstack-operators | 01:54 | |
*** david-lyle has joined #openstack-operators | 02:08 | |
*** Marga_ has quit IRC | 02:10 | |
*** Marga_ has joined #openstack-operators | 02:11 | |
david-lyle | r1chardj0n3s: is https://review.openstack.org/#/c/141273 your highest priority patch to merge? | 02:27 |
---|---|---|
*** Marga_ has quit IRC | 02:33 | |
*** david-lyle has quit IRC | 02:46 | |
*** harlowja is now known as harlowja_away | 02:46 | |
*** david-lyle has joined #openstack-operators | 02:50 | |
*** signed8bit_ZZZzz has quit IRC | 02:54 | |
*** blair has quit IRC | 03:02 | |
*** blair has joined #openstack-operators | 03:09 | |
*** Marga_ has joined #openstack-operators | 03:25 | |
*** david-lyle has quit IRC | 03:31 | |
*** VW_ has joined #openstack-operators | 03:32 | |
*** Marga_ has quit IRC | 04:03 | |
*** Marga_ has joined #openstack-operators | 04:04 | |
*** blair has quit IRC | 04:09 | |
*** david-lyle has joined #openstack-operators | 04:09 | |
*** wabu has quit IRC | 04:42 | |
*** wabu has joined #openstack-operators | 04:43 | |
*** avozza is now known as zz_avozza | 04:55 | |
*** zz_avozza is now known as avozza | 04:56 | |
*** klindgren__ has joined #openstack-operators | 05:02 | |
*** klindgren_ has quit IRC | 05:05 | |
*** sanjayu has joined #openstack-operators | 05:12 | |
*** blair has joined #openstack-operators | 05:29 | |
*** sanjayu has quit IRC | 05:44 | |
*** alop has quit IRC | 05:46 | |
*** blair has quit IRC | 05:59 | |
*** sanjayu has joined #openstack-operators | 06:10 | |
*** blair has joined #openstack-operators | 06:11 | |
*** VW_ has quit IRC | 06:12 | |
*** blair has quit IRC | 06:45 | |
*** markvoelker_ has quit IRC | 07:28 | |
*** Miouge has joined #openstack-operators | 07:40 | |
*** avozza is now known as zz_avozza | 07:44 | |
*** belmoreira has joined #openstack-operators | 07:51 | |
*** blair has joined #openstack-operators | 07:57 | |
*** blair has quit IRC | 08:09 | |
*** zz_avozza is now known as avozza | 08:16 | |
*** blair has joined #openstack-operators | 08:36 | |
*** blair has quit IRC | 08:47 | |
*** belmoreira has quit IRC | 08:51 | |
*** bvandenh has joined #openstack-operators | 09:13 | |
*** derekh has joined #openstack-operators | 09:22 | |
*** belmoreira has joined #openstack-operators | 09:55 | |
*** blair has joined #openstack-operators | 10:04 | |
*** blair has quit IRC | 10:21 | |
*** Miouge_ has joined #openstack-operators | 11:48 | |
*** Miouge has quit IRC | 11:49 | |
*** Miouge_ is now known as Miouge | 11:49 | |
*** Miouge has quit IRC | 11:52 | |
*** reed has joined #openstack-operators | 11:55 | |
*** Miouge has joined #openstack-operators | 12:24 | |
*** Miouge has quit IRC | 12:32 | |
*** Miouge has joined #openstack-operators | 12:49 | |
*** markvoelker has joined #openstack-operators | 12:51 | |
*** VW_ has joined #openstack-operators | 13:22 | |
*** VW_ has quit IRC | 13:23 | |
*** sanjayu has quit IRC | 14:23 | |
*** radez is now known as radez_g0n3 | 14:25 | |
*** signed8bit has joined #openstack-operators | 14:26 | |
*** j05h has joined #openstack-operators | 14:26 | |
*** j05h has quit IRC | 14:28 | |
*** j05h has joined #openstack-operators | 14:29 | |
*** ctina has joined #openstack-operators | 14:30 | |
*** signed8bit has quit IRC | 14:32 | |
*** signed8bit has joined #openstack-operators | 14:37 | |
*** VW_ has joined #openstack-operators | 14:42 | |
*** Piet has quit IRC | 15:18 | |
*** mdorman has joined #openstack-operators | 15:22 | |
*** j05h has quit IRC | 15:26 | |
*** beddari has quit IRC | 15:36 | |
*** beddari has joined #openstack-operators | 15:39 | |
*** Piet has joined #openstack-operators | 15:40 | |
*** emagana has joined #openstack-operators | 15:44 | |
*** beddari has quit IRC | 15:53 | |
*** j05h has joined #openstack-operators | 16:15 | |
*** Piet has quit IRC | 16:17 | |
*** radez_g0n3 is now known as radez | 16:25 | |
*** klindgren__ is now known as klindgren | 16:26 | |
*** VW_ has quit IRC | 16:29 | |
*** j05h has quit IRC | 16:29 | |
*** VW_ has joined #openstack-operators | 16:30 | |
*** VW_ has quit IRC | 16:34 | |
*** j05h has joined #openstack-operators | 16:41 | |
*** Marga_ has quit IRC | 16:47 | |
*** dmsimard_away is now known as dmsimard | 16:47 | |
*** Piet has joined #openstack-operators | 16:48 | |
*** VW_ has joined #openstack-operators | 16:56 | |
*** SimonChung has joined #openstack-operators | 17:01 | |
*** alop has joined #openstack-operators | 17:09 | |
*** ctina has quit IRC | 17:11 | |
*** ctina has joined #openstack-operators | 17:14 | |
*** derekh has quit IRC | 17:25 | |
*** cpschult has joined #openstack-operators | 17:40 | |
*** Marga_ has joined #openstack-operators | 17:47 | |
*** harlowja_away is now known as harlowja | 17:47 | |
mfisch | may have asked this before, but does anyone here at all use heat? | 17:54 |
jlk | I think we have it in our offering, but we may not have it enabled for anybody | 17:57 |
jlk | we don't use it ourselves, that's what Ansible is for :) | 17:57 |
mdorman | same here, we have it set up but pretty sure nobody uses it | 17:58 |
*** Piet has quit IRC | 18:00 | |
mdorman | jlk: along the lines of the recent thread about cafile setting… do you know off the top of your head where/how the cafile should be configured such that the command line tools will use the proper cafile for validation? specifically the keystone cli tool is what i’m fighting against now | 18:01 |
jlk | good question! | 18:01 |
mdorman | hah | 18:01 |
mdorman | --os_cacert on the command line. but how to put that in a conf file | 18:03 |
*** SimonChung has quit IRC | 18:03 | |
*** SimonChung1 has joined #openstack-operators | 18:03 | |
jlk | OS_CACERT=/etc/ssl/my-root-cert.pem | 18:03 |
jlk | oh in a conf file... | 18:03 |
jlk | didn't know they read from a conf | 18:03 |
mdorman | yeah maybe they don't | 18:06 |
mdorman | see, it’s supposed to default ot using the system ca bundle | 18:06 |
mdorman | which i thought was /etc/pki/tls/certs/ca-bundle.crt | 18:06 |
mdorman | i mean that’s what curl uses. but i think urllib3 uses something else. b/c curl works, but python clients don't | 18:06 |
klindgren | strace! | 18:07 |
mdorman | good idea kris | 18:09 |
mdorman | it’d be pretty frustrating if this rquires another command line argument. b/c then it’s going to require retooing all the keystone resource puppet providers | 18:09 |
*** SimonChung has joined #openstack-operators | 18:18 | |
*** SimonChung1 has quit IRC | 18:18 | |
jlk | ah | 18:18 |
jlk | python on some platforms has their own CA bundle | 18:18 |
jlk | in some of our set ups, we were symlinkin python requests's CA bundle to the system CA bundle, /etc/ssl/certs/ca-certificates or something. Now we just specifically point to our CA file in API configs | 18:19 |
jlk | mdorman: how are you passing along auth details? User/pass/tenant? | 18:19 |
mdorman | i believe in teh puppet providers it just sticks tthe user and pass on the command line | 18:22 |
mdorman | yeah i suspect python is using it’s own ca bundle | 18:22 |
mdorman | so thinking along those same lines, figure out where that is and symlink | 18:22 |
*** bvandenh has quit IRC | 18:23 | |
verdurin | There's an [ssl] section in keystone.conf where you can configure this | 18:24 |
verdurin | certfile=, keyfile=, ca_certs= etc. | 18:25 |
mdorman | verdurin: right, but when running the keystone command line tool, it only grabs options from the command line, not from the conf file (as far as i can tell) | 18:26 |
mdorman | fyi /usr/lib/python2.7/site-packages/requests/cacert.pem ftw. dropped my other certs in there, and success! | 18:27 |
jlk | verdurin: that's not exactly the same | 18:27 |
jlk | verdurin: keystone client doesn't read keystone.conf file | 18:28 |
*** SimonChung1 has joined #openstack-operators | 18:29 | |
*** SimonChung has quit IRC | 18:29 | |
*** emagana has quit IRC | 18:30 | |
*** SimonChung has joined #openstack-operators | 18:30 | |
*** SimonChung1 has quit IRC | 18:30 | |
*** emagana has joined #openstack-operators | 18:30 | |
*** SimonChung has quit IRC | 18:30 | |
*** SimonChung1 has joined #openstack-operators | 18:30 | |
*** david-lyle is now known as david-lyle_afk | 18:31 | |
*** emagana has quit IRC | 18:31 | |
*** emagana has joined #openstack-operators | 18:31 | |
*** avozza is now known as zz_avozza | 18:33 | |
*** Piet has joined #openstack-operators | 18:39 | |
*** Piet has quit IRC | 18:42 | |
*** Miouge has quit IRC | 18:44 | |
*** Piet has joined #openstack-operators | 18:54 | |
*** SimonChung has joined #openstack-operators | 18:56 | |
*** SimonChung1 has quit IRC | 18:56 | |
*** emagana has quit IRC | 18:58 | |
*** emagana has joined #openstack-operators | 18:59 | |
*** emagana has quit IRC | 19:02 | |
*** emagana has joined #openstack-operators | 19:03 | |
*** belmoreira has quit IRC | 19:03 | |
klindgren | Did anyone ever figure out why with neutron you must install every plugin just to do a db sync? | 19:04 |
*** VW__ has joined #openstack-operators | 19:04 | |
*** VW__ has quit IRC | 19:04 | |
klindgren | Seem's crazy to install everything - just to uninstall it later. | 19:05 |
*** VW__ has joined #openstack-operators | 19:05 | |
*** VW_ has quit IRC | 19:05 | |
jlk | what? | 19:06 |
jlk | no... | 19:06 |
jlk | we only have the ml2 plugin installed and that's what we use | 19:06 |
*** zz_avozza is now known as avozza | 19:06 | |
jlk | we reference neutron.conf and ml2 config in the neutron-manage call | 19:06 |
klindgren | from icehouse -> juno | 19:11 |
klindgren | for db-upgrade | 19:11 |
jlk | yeah | 19:11 |
jlk | we do havana -> juno that way | 19:11 |
klindgren | it would blow up if we didn't have all hte plugins installed | 19:11 |
jlk | that's... odd | 19:11 |
jlk | https://github.com/blueboxgroup/ursula/blob/master/roles/neutron-control/tasks/main.yml#L25 | 19:12 |
jlk | We do this just before as well https://github.com/blueboxgroup/ursula/blob/master/upgrade.yml#L234 | 19:12 |
*** Piet has quit IRC | 19:14 | |
klindgren | neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file <your plugin config file> current | 19:15 |
klindgren | neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file <your plugin config file> upgrade juno | 19:15 |
*** signed8bit is now known as signed8bit_ZZZzz | 19:15 | |
klindgren | is what we are doing | 19:15 |
*** signed8bit_ZZZzz is now known as signed8bit | 19:15 | |
jlk | current just reports things, doesn't actually /do/ anything | 19:15 |
klindgren | but that command only works if we install all the plugins - like bigswitch, brocade, cisco, ovs, ect ect | 19:15 |
*** Piet has joined #openstack-operators | 19:16 | |
jlk | anybody know where in nova code the services table gets created, populated? | 19:31 |
mfisch | klindgren: we also dont have that problem | 19:35 |
mfisch | klindgren: we also stamped like jlk said, we did that weeks ago, | 19:36 |
*** VW__ has quit IRC | 19:36 | |
*** Marga_ has quit IRC | 19:41 | |
*** VW_ has joined #openstack-operators | 19:41 | |
klindgren | hrm | 19:43 |
klindgren | you guys are using ubuntu packages? | 19:44 |
jlk | no | 19:46 |
jlk | .... and now I find myself reading up on how python implements socket.gethostname() | 19:46 |
klindgren | just apply the glibc updates and reboot ;-) | 19:47 |
jlk | not for that | 19:47 |
jlk | Trying to determine impact of changing the hostname of a system | 19:47 |
jlk | on some systems the short hostname was defined as foo.bar | 19:47 |
jlk | we want to change it so that shortname is foo, full name is foo.bar | 19:47 |
jlk | but I don't know how python is getting the hostname that it openstack uses for the services tables and other things | 19:48 |
mfisch | klindgren: we are using ubuntu packages | 19:50 |
klindgren | jlk - you are using venv's? | 19:50 |
jlk | klindgren: we will soon, but right now it's git clones + pip installs | 19:51 |
*** SimonChung1 has joined #openstack-operators | 19:51 | |
*** SimonChung has quit IRC | 19:51 | |
*** david-lyle_afk is now known as david-lyle | 19:51 | |
dvorak | mdorman: the puppet providers all pass username/password/etc via the environment | 19:58 |
dvorak | jlk: last I looked into the migrations, there is some conditional code in there around which plugins are configured. maybe that's related | 19:59 |
*** emagana has quit IRC | 20:03 | |
*** cdelatte has joined #openstack-operators | 20:05 | |
klindgren | wondering if I am missing a specifc config directive in the neutron.conf | 20:05 |
*** cdelatte has quit IRC | 20:06 | |
*** cdelatte has joined #openstack-operators | 20:06 | |
*** cdelatte has quit IRC | 20:07 | |
*** cdelatte has joined #openstack-operators | 20:07 | |
*** emagana has joined #openstack-operators | 20:09 | |
jlk | klindgren: what do you have for core_plugin ? | 20:11 |
klindgren | core_plugin=neutron.plugins.ml2.plugin.Ml2Plugin | 20:12 |
*** Marga_ has joined #openstack-operators | 20:12 | |
jlk | hrm. | 20:12 |
jlk | yeah | 20:12 |
*** emagana has quit IRC | 20:13 | |
*** Marga_ has quit IRC | 20:14 | |
*** Marga_ has joined #openstack-operators | 20:14 | |
*** Marga_ has quit IRC | 20:18 | |
*** emagana has joined #openstack-operators | 20:22 | |
dvorak | jlk: have you stamped the db? | 20:31 |
dvorak | ours wasn't stamped, I think because of a bug in the puppet modules | 20:31 |
jlk | dvorak: we stamp ours as part of the upgrade from havana yes. | 20:33 |
jlk | after that we won't stamp it | 20:33 |
dvorak | well, in theory you shouldn't ever need to | 20:33 |
dvorak | but yeah, that's how we did ours too | 20:33 |
jlk | yeah, theory. However | 20:33 |
jlk | the upgrade guides tell you to stamp it, so... | 20:33 |
dvorak | I was just wondering, because when I hadn't stamped ours, it exploded when I ran upgrade :) | 20:33 |
dvorak | bah, who reads the upgrade guide? | 20:34 |
dvorak | actually, I don't think there is an upgrade guide for juno | 20:34 |
dvorak | which seems kind of crazy | 20:34 |
jlk | sure there is | 20:34 |
jlk | https://wiki.openstack.org/wiki/ReleaseNotes/Juno | 20:35 |
jlk | https://wiki.openstack.org/wiki/ReleaseNotes/Juno#Upgrade_Notes_6 | 20:35 |
dvorak | apparently you're right. I couldn't easily find it when I looked a few weeks ago | 20:36 |
dvorak | we did our upgrade mostly based on the release notes and experimentation | 20:36 |
jlk | I've stared at them a lot over the past month or so | 20:36 |
dvorak | eh, I wouldn't call that an upgrade guide :) | 20:36 |
jlk | upgrade hints :) | 20:37 |
dvorak | I found this - https://git.openstack.org/cgit/openstack/operations-guide/commit/?id=ae8d4b91c2f60d427bdde57acc3f1b769cca393f | 20:37 |
dvorak | which translates to this url - http://docs.openstack.org/openstack-ops/content/upgrade-icehouse-juno.html | 20:38 |
jlk | rendered as http://docs.openstack.org/openstack-ops/content/upgrade-icehouse-juno.html | 20:38 |
dvorak | heh | 20:38 |
dvorak | it would have been pretty handy to find that before we did the upgrade | 20:38 |
jlk | looks like some extra steps in here that are unnecessary | 20:39 |
dvorak | well, and for us, moving to the master branch for the stackforge puppet modules did most of that | 20:40 |
jlk | there's a gotcha as well, don't go from UUID to PKI tokens in keystone, otherwise you'll have to restart all your services ... again. | 20:40 |
dvorak | well, I think PKI was the default in icehouse, no? | 20:40 |
jlk | we're doing our upgrade one service at a time, keeping the whole thing operational minus that one service as we go. | 20:40 |
jlk | dvorak: yeah, might have been. We''re coming from Havana, which a lot of places appear to still be on | 20:40 |
dvorak | we did everything in one shot, one site last night, the other tomorrow night | 20:40 |
jlk | (looking at the openstack marketplace) | 20:41 |
dvorak | yeah, I think it'd be interesting to see how many people are running what versions | 20:41 |
jlk | we have like 30 sites to do, it'll be a bit slower to get them all done, but we don't want full outage | 20:41 |
dvorak | seems unavoidable given the db migrations required | 20:41 |
jlk | just per-service | 20:41 |
jlk | you can do nova stuff that doesn't involve glance while you upgrade glance | 20:42 |
dvorak | our outage was like 15 minutes | 20:42 |
dvorak | most of that was package installations | 20:42 |
jlk | we also keep instances reachable and operational during the upgrade | 20:42 |
dvorak | we would have kept instances reachable if someone (me) hadn't accidently added openvswitch-switch to the list of services to shutdown | 20:43 |
jlk | hah | 20:43 |
dvorak | it's stupid, but it's a list of like twenty services to shut down, and like 3 other people reviewed the list. unfortunately some screwups are bound to get through sometimes | 20:44 |
dvorak | thankfully we realized what was going on quickly and had that fixed in about 10 minutes | 20:45 |
*** emagana has quit IRC | 20:46 | |
*** emagana has joined #openstack-operators | 20:47 | |
*** emagana has quit IRC | 20:48 | |
*** emagana has joined #openstack-operators | 20:48 | |
*** Piet has quit IRC | 21:00 | |
*** Piet has joined #openstack-operators | 21:03 | |
*** Piet has quit IRC | 21:14 | |
*** Marga_ has joined #openstack-operators | 21:14 | |
*** Piet has joined #openstack-operators | 21:16 | |
*** j05h has quit IRC | 21:17 | |
*** Marga_ has quit IRC | 21:18 | |
*** j05h has joined #openstack-operators | 21:19 | |
*** Marga_ has joined #openstack-operators | 21:49 | |
*** blair has joined #openstack-operators | 21:50 | |
*** blair has quit IRC | 21:55 | |
*** blair has joined #openstack-operators | 21:57 | |
*** ctina has quit IRC | 22:02 | |
*** j05h has quit IRC | 22:06 | |
*** radez is now known as radez_g0n3 | 22:16 | |
*** j05h has joined #openstack-operators | 22:25 | |
*** blair has quit IRC | 22:33 | |
*** SimonChung has joined #openstack-operators | 22:38 | |
*** SimonChung1 has quit IRC | 22:38 | |
*** blair has joined #openstack-operators | 22:40 | |
*** emagana has quit IRC | 22:52 | |
*** Rockyg has joined #openstack-operators | 23:00 | |
*** emagana has joined #openstack-operators | 23:00 | |
*** SimonChung1 has joined #openstack-operators | 23:01 | |
*** SimonChung has quit IRC | 23:01 | |
*** cpschult has quit IRC | 23:05 | |
*** Rockyg has quit IRC | 23:20 | |
*** SimonChung has joined #openstack-operators | 23:23 | |
*** SimonChung1 has quit IRC | 23:23 | |
*** VW_ has quit IRC | 23:33 | |
*** reed has quit IRC | 23:43 | |
*** dmsimard is now known as dmsimard_away | 23:56 | |
*** mdorman has quit IRC | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!