| *** saurabhs has quit IRC | 00:51 | |
| *** witlessb has quit IRC | 01:09 | |
| *** russellb has quit IRC | 01:09 | |
| *** russellb has joined #openstack-sahara | 01:15 | |
| *** openstackgerrit has joined #openstack-sahara | 01:30 | |
| *** witlessb has joined #openstack-sahara | 02:56 | |
| *** witlessb has quit IRC | 03:01 | |
| *** openstackgerrit has quit IRC | 03:10 | |
| *** anteaya has quit IRC | 03:37 | |
| *** anteaya has joined #openstack-sahara | 03:45 | |
| *** chandankumar has joined #openstack-sahara | 03:57 | |
| *** chandankumar has quit IRC | 05:36 | |
| *** chandankumar has joined #openstack-sahara | 05:51 | |
| *** k4n0 has joined #openstack-sahara | 06:37 | |
| *** sreshetnyak has quit IRC | 08:06 | |
| *** ylobankov has quit IRC | 08:07 | |
| *** ylobankov has joined #openstack-sahara | 08:19 | |
| *** sreshetnyak has joined #openstack-sahara | 08:19 | |
| *** stannie1 has joined #openstack-sahara | 08:27 | |
| *** IvanBerezovskiy has joined #openstack-sahara | 09:35 | |
| *** NikitaKonovalov has quit IRC | 09:51 | |
| *** SergeyLukjanov has quit IRC | 09:51 | |
| *** aignatov has quit IRC | 09:51 | |
| *** alazarev has quit IRC | 09:51 | |
| *** dmitryme has quit IRC | 09:51 | |
| *** ruhe has quit IRC | 09:51 | |
| *** aignatov has joined #openstack-sahara | 09:52 | |
| *** dmitryme has joined #openstack-sahara | 09:52 | |
| *** alazarev has joined #openstack-sahara | 09:52 | |
| *** NikitaKonovalov has joined #openstack-sahara | 09:53 | |
| *** ruhe has joined #openstack-sahara | 09:53 | |
| *** SergeyLukjanov has joined #openstack-sahara | 09:53 | |
| *** witlessb has joined #openstack-sahara | 10:20 | |
| *** stannie1 has quit IRC | 10:29 | |
| *** tosky has joined #openstack-sahara | 11:00 | |
| *** witlessb has quit IRC | 11:02 | |
| *** tellesnobrega has joined #openstack-sahara | 11:33 | |
| *** witlessb has joined #openstack-sahara | 11:39 | |
| *** witlessb has quit IRC | 11:54 | |
| *** miqui has joined #openstack-sahara | 12:15 | |
| *** _crobertsrh is now known as crobertsrh | 12:57 | |
| *** witlessb has joined #openstack-sahara | 13:03 | |
| *** tmckay1 has quit IRC | 13:08 | |
| *** dboik has joined #openstack-sahara | 13:24 | |
| *** boris-42 has joined #openstack-sahara | 13:25 | |
| *** dboik has quit IRC | 13:25 | |
| *** dboik has joined #openstack-sahara | 13:26 | |
| *** witlessb has quit IRC | 13:49 | |
| *** tnovacik|gone has joined #openstack-sahara | 13:55 | |
| *** tmckay has joined #openstack-sahara | 13:59 | |
| *** coolsvap has quit IRC | 14:02 | |
| *** chandankumar has quit IRC | 14:05 | |
| *** witlessb has joined #openstack-sahara | 14:06 | |
| tmckay | zhidong, ping | 14:11 |
|---|---|---|
| tmckay | zhidong, saw your email on openstack-dev, I thought you might be here | 14:12 |
| crobertsrh | Yeah, I noticed the email too | 14:12 |
| tmckay | we should hit on dockerization -- what it means for Sahara at the summit | 14:12 |
| elmiko | seems like that is going to be an issue for us | 14:12 |
| tmckay | Worthy stuff for kilo, I think | 14:12 |
| crobertsrh | Sure, might be a good topic for the meetup | 14:13 |
| elmiko | don't we use hostname for identification of nodes and addressing? | 14:13 |
| tmckay | in fact, I think it might be good to keep EDP work small and focus on stuff like docker | 14:13 |
| elmiko | we definitely should investigate having someone reach out to the kolla group | 14:14 |
| crobertsrh | a kolla holla? | 14:14 |
| tmckay | elmiko, yeah. We'll have to dig in to the sequence. At some point, we've got an IP. That should be enough without a hostname. If there was a way to pre-assign hostnames, and then read them back from the IP, Sahara could just receive the host names | 14:14 |
| elmiko | i hung out for one of their meetings and they are just about to implement a sahara container | 14:14 |
| elmiko | crobertsrh: yes! | 14:14 |
| tmckay | is kolla "docker for openstack"? | 14:14 |
| elmiko | tmckay: kinda yea, it is a spin-off of triple-o to implement a stack in docker containers | 14:15 |
| crobertsrh | Sahara -- just *try* to contain us | 14:15 |
| elmiko | lol nice | 14:15 |
| tosky | can we have a proper HA mode before fancy-trandy-stuff like docker? | 14:16 |
| elmiko | tosky: that's a good point | 14:17 |
| tmckay | man, I need to learn more projects | 14:19 |
| elmiko | i hear summit is a good place for that ;) | 14:19 |
| tmckay | I need to learn how not to sleep | 14:20 |
| elmiko | lol | 14:20 |
| elmiko | that's a toughie | 14:20 |
| tmckay | or train my kids to be openstack engineers, and I can manage them | 14:20 |
| elmiko | oh man... imagine the power of having your own in-house dev team! | 14:21 |
| tmckay | so far nobody has shown any interest | 14:22 |
| elmiko | oh well | 14:22 |
| elmiko | dad's job is just not cool enough ;P | 14:22 |
| tmckay | Some asked one of them what I did, the answer was "pushes buttons" | 14:23 |
| tmckay | they were young at the time | 14:23 |
| elmiko | LOL | 14:23 |
| elmiko | that's priceless | 14:23 |
| *** k4n0 has quit IRC | 14:34 | |
| *** miqui has quit IRC | 14:51 | |
| *** tmckay has quit IRC | 15:17 | |
| *** dboik has quit IRC | 15:46 | |
| *** tmckay has joined #openstack-sahara | 16:04 | |
| *** ylobankov has quit IRC | 16:26 | |
| *** chandankumar has joined #openstack-sahara | 16:30 | |
| SergeyLukjanov | tmckay, could you please take a look at https://review.openstack.org/#/c/115259/6 and approve if it's ok for you | 16:55 |
| *** crobertsrh has quit IRC | 16:58 | |
| jodah | was looking through the storm blueprint - curious, has deploying zookeeper as a separate cluster (via a separate plugin perhaps) ever come up? | 17:00 |
| jodah | ...since zookeeper clusters are often shared across many services, including but not limited to storm | 17:00 |
| elmiko | jodah: i don't think so, we've recently implemented the ability for hdp clusters to have multiple zks, but i don't think we've discussed cross-cluster things like this. | 17:01 |
| elmiko | it's a cool idea though | 17:01 |
| jodah | reason i mention is i see zookeeper as being a separate thing from deploying storm. certainly it's a dependency, but it also could be fairly separate | 17:01 |
| jodah | for example, we have a few storm and kafka clusters that all share the same zookeeper cluster | 17:02 |
| elmiko | yea, it's a good idea. i'm wondering how we would fit it into sahara's model. good idea for discussion, can you make our meeting today? | 17:03 |
| jodah | yep | 17:03 |
| elmiko | cool, definitely bring it up | 17:03 |
| *** IvanBerezovskiy has left #openstack-sahara | 17:09 | |
| tmckay | SergeyLukjanov, sure, I was out for car repairs | 17:12 |
| *** tellesnobrega has quit IRC | 17:18 | |
| tmckay | elmiko, my take on https://review.openstack.org/#/c/115259/6 is that there is no additional security risk in putting swift auth info in hive-site.xml, node is still locked down. I assume from your +1 that you agree. | 17:23 |
| elmiko | tmckay: well yea, for now. the way oikawa described it, it sounds similar to things we do with proxy users and workflow.xml. | 17:24 |
| elmiko | what makes me nervous is that the creds for a proxy will exist for the duration of the cluster, which could pose a vulnerability | 17:25 |
| tmckay | yeah. And I'm going to need to write swift auth info to spark job dirs, too. | 17:25 |
| elmiko | much like you, i'm thinking we might be able to leverage barbican to help with the secret management | 17:25 |
| tmckay | elmiko, I think when we talked about this before, we said that we considered the node itself secure and so information written there was okay | 17:26 |
| elmiko | tmckay: yea | 17:26 |
| tmckay | if you give out the ssh key to get into the cluster, well ... shame on you | 17:26 |
| *** stannie has quit IRC | 17:26 | |
| elmiko | right, we are no worse off, and slightly better actually, if we use the proxies. i'm still nervous about the storage on nodes, but we're making steps to improve. | 17:27 |
| tmckay | how could we get around storing on the node? | 17:28 |
| * tmckay forgets | 17:28 | |
| elmiko | tmckay: sec, i'm riding too many irc chans lol | 17:28 |
| tmckay | heh, okay | 17:28 |
| tmckay | SergeyLukjanov, done | 17:28 |
| elmiko | tmckay: ok, so my thoughts on secrets and nodes and whatnot | 17:46 |
| elmiko | tmckay: basically, i want to see if we can leverage barbican to perform something akin to: | 17:46 |
| elmiko | 1. sahara generates proxy user creds and stores them in barbican | 17:46 |
| elmiko | 2. sahara registers the nodes as consumers of those secrets with barbican | 17:47 |
| elmiko | 3. during operation, the nodes will be able to pull their creds from barbican, and keep them in non-text form | 17:47 |
| elmiko | the big downside i see to all this, is the hadoop integration. | 17:47 |
| tmckay | nice. How does #2 work? | 17:47 |
| *** chandankumar has quit IRC | 17:48 | |
| elmiko | what i'm hoping is that during #2, sahara would take the public keys from all nodes and register them with barbican to allow those nodes to use their own 2-part keys for encryption. | 17:48 |
| elmiko | thus, barbican would have the public keys to allow encrypting back to the nodes | 17:49 |
| tmckay | oh, nice | 17:49 |
| elmiko | i don't have all the details worked out yet, but i think something along these lines should be possible | 17:49 |
| elmiko | like i said though, the biggest difficulty will be integrating with hadoop, spark, etc... | 17:49 |
| tmckay | sounds very cool. | 17:50 |
| elmiko | i'm gonna try and talk it through with the ossg folks at summit, i know they are doing a ton of work with barbican currently | 17:51 |
| elmiko | just saw the email from croberts, that sux =( | 17:51 |
| SergeyLukjanov | meeting in a few mins | 17:58 |
| *** sreshetnyak has quit IRC | 18:00 | |
| *** sreshetnyak has joined #openstack-sahara | 18:00 | |
| tosky | oh, daylight saving time, I forgot | 18:02 |
| *** dboik has joined #openstack-sahara | 18:05 | |
| *** crobertsrh has joined #openstack-sahara | 18:21 | |
| *** chandankumar has joined #openstack-sahara | 18:33 | |
| *** chandankumar has quit IRC | 18:37 | |
| jodah | crobertsrh which image were you booting with docker yesterday? | 19:23 |
| crobertsrh | Successfully with the default cirros image | 19:23 |
| crobertsrh | unsuccessfully with a custom image I built with our disk image builder tool | 19:24 |
| jodah | this was just using the docker VIRT_DRIVER on devstack? | 19:24 |
| crobertsrh | Yeah | 19:24 |
| jodah | i haven't tried the image builder yet - what was the basis for the image you built? | 19:25 |
| crobertsrh | vanilla plugin version 1 (fedora) | 19:25 |
| crobertsrh | The image runs fine with docker directly, but has issues when using the nova-docker driver | 19:26 |
| crobertsrh | I found at least one bug in nova-docker. They gave me a patch for that this morning, but I'm still unsuccessful. | 19:26 |
| jodah | what did you have to do when building the image to get it built for docker? | 19:28 |
| crobertsrh | I tweak the diskimage-create.sh script to use the "-t tar" flag when it calls the base project's build script | 19:29 |
| crobertsrh | It's not a Dockerfile based build. It essentially just creates a tarball of our regular image. | 19:30 |
| *** tosky has quit IRC | 19:31 | |
| tmckay | SergeyLukjanov, looks like only you have +2 on https://review.openstack.org/#/c/130596/ | 20:09 |
| tmckay | stable/juno, I guess | 20:09 |
| *** tnovacik|gone has quit IRC | 20:29 | |
| SergeyLukjanov | tmckay, yup | 20:43 |
| SergeyLukjanov | tmckay, the stable/* branchs maintained by the stable maintenance team | 20:44 |
| tmckay | ah, I didn't know about that | 20:44 |
| SergeyLukjanov | and I'm become member of this team, so, we have at least one +2 from our project | 20:44 |
| tmckay | makes sense | 20:44 |
| SergeyLukjanov | it's only for integrated projects | 20:44 |
| jodah | crobertsrh so on diskimage-create.sh, where does the base project's build script get called, where you inserted -t tar ? | 21:09 |
| crobertsrh | let me find the lines | 21:09 |
| jodah | tnx | 21:09 |
| crobertsrh | There is a line like: disk-image-create $fedora_elements_sequence -o $fedora_image_name | 21:10 |
| crobertsrh | stick a -t tar before the -o | 21:11 |
| crobertsrh | Oh...also, you can change the "mv" line that follows it | 21:11 |
| crobertsrh | to mv the ".tar" file rather than .qcow2 | 21:11 |
| crobertsrh | otherwise that line will fail and you'll have to hunt the .tar file down | 21:11 |
| jodah | thanks - giving this a try | 21:12 |
| jodah | what was the fix the nova guys suggested? | 21:13 |
| crobertsrh | Well, it was http://paste.openstack.org/show/127046/ but I now get a 500 from past.o.o | 21:13 |
| crobertsrh | paste.o.o, that is | 21:13 |
| crobertsrh | oh, it worked now | 21:14 |
| crobertsrh | I think nova may have also just made a change to use oslo.concurrency.processutils rather than nova.openstack.common.processutils, so depending on when you pulled your nova bits, you might need to change that import in a couple of places in nova-docker | 21:15 |
| jodah | i just created a new devstack | 21:16 |
| crobertsrh | Ok. If your nova-cpu process doesn't want to start, you probably need the processutils change....at least, I did | 21:17 |
| crobertsrh | I'm still unable to have my stack launch the sahara container though. | 21:17 |
| crobertsrh | The guys in #nova-docker were helpful though, so hopefully they can help a bit more :) | 21:17 |
| *** sreshetnyak has quit IRC | 21:22 | |
| *** crobertsrh is now known as _crobertsrh | 21:43 | |
| *** coolsvap has joined #openstack-sahara | 22:34 | |
| *** witlessb has quit IRC | 23:08 | |
| *** boris-42 has quit IRC | 23:45 | |
| *** boris-42 has joined #openstack-sahara | 23:51 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!