Monday, 2016-01-11

*** tzn has joined #openstack-solar00:23
*** tzn has quit IRC00:28
*** tzn has joined #openstack-solar01:24
*** tzn has quit IRC01:29
*** tzn has joined #openstack-solar02:25
*** tzn has quit IRC02:31
*** tzn has joined #openstack-solar04:27
*** tzn has quit IRC04:32
*** tzn has joined #openstack-solar06:29
*** tzn has quit IRC06:34
*** dshulyak_ has joined #openstack-solar07:25
*** tzn has joined #openstack-solar07:30
*** tzn has quit IRC07:34
*** tzn has joined #openstack-solar08:31
*** openstackgerrit has quit IRC08:32
*** openstackgerrit has joined #openstack-solar08:33
*** tzn has quit IRC08:36
*** salmon_ has joined #openstack-solar09:11
openstackgerritMerged openstack/solar: Fixed OSError during concurrent TempFileHandler init
pigmejsalmon_: all except torrent are ok09:25
salmon_we will see ;)09:25
pigmejtorrent is:09:26
pigmej1) not rebased with master09:26
pigmej2) some weird things happen there09:26
salmon_I know, it didn't work 3 days ag o :P09:31
pigmejI'm not having keyerror;p09:31
pigmejfor some reason it just doesn't work properly in container..09:35
*** tzn has joined #openstack-solar09:36
openstackgerritJedrzej Nowak proposed openstack/solar: Fixed torrent transport
pigmejI have no idea currently what's wrong with it inside container, It works flawlessly outside09:40
openstackgerritMerged openstack/solar: Use stevedore for transports management
salmon_pigmej: rebase !09:57
pigmejsalmon_: doing10:03
openstackgerritJedrzej Nowak proposed openstack/solar: Standarized props names in ssh transports
openstackgerritJedrzej Nowak proposed openstack/solar: Ansible Playbook uses now solar_transports
openstackgerritJedrzej Nowak proposed openstack/solar: Implements ansible_playbook_local handler
pigmejsalmon_: rebased10:12
pigmejOk, I'm going to check why the heck torrent is not working in container10:36
*** tzn has quit IRC10:58
openstackgerritJedrzej Nowak proposed openstack/solar: Raise better error when requested transport is not found
*** salmon_ has quit IRC11:22
*** salmon_ has joined #openstack-solar11:23
dshulyak_pigmej: is it possible to detect 2i support in a different way ?12:22
pigmejdshulyak_: "depends" :)12:24
pigmejthere are several ways to do so but I wonder what wil be the best there12:24
dshulyak_i wasnt able to find any info in bucket properties12:24
pigmejdshulyak_: we have this problem only in delete ?12:25
dshulyak_pigmej: yes, bitcask backend used only for single model, which is Lock12:27
pigmejok, so let's make it like this now but maybe add todo there ?12:27
pigmejbecause get_keys is really slow12:27
pigmejand it's kinda "the most stupid method for whole cluster"12:27
pigmejit scans whole key space, and then filters keys for bucket...12:28
pigmejso maybe instaed please make sure that we will not execute it on 2i enabled buckets ?12:28
pigmejafair riak client throws something like 'secondary index is not supported with ... backend' right?12:28
pigmejdoes it make sense for you dshulyak_ ?12:29
pigmejyou could use also your bucket_properties dict12:29
pigmejor how it was called12:29
dshulyak_pigmej: i will check message, but what about bucket_properties? there is no information about 2i support12:30
pigmejsomewhere is the info about backend used for this bucket12:33
pigmejonly one with 2i is leveldb12:33
pigmejI cannot find my code which did it right now12:33
pigmejbut I meant your bucket_properties dict in python code12:33
*** tzn has joined #openstack-solar12:42
openstackgerritDmitry Shulyak proposed openstack/solar: Implement riak based lock mechanism for orchestration
openstackgerritMerged openstack/solar: Standarized props names in ssh transports
openstackgerritJedrzej Nowak proposed openstack/solar: Ansible Playbook uses now solar_transports
pigmejdshulyak_: we will be able to use your cool playbook handler :))12:52
pigmejsalmon_: I have to admit, that it's different than ansible_template but it's more clever :)12:52
dshulyak_pigmej: you are talking about ansible_local?12:53
dshulyak_or ansible_playbook but with our transports?12:53
pigmejdshulyak_: yup :)12:53
pigmejthe second12:53
pigmejansible_local is basically your thingy12:53
pigmejdshulyak_: We will have ansible + ansible_local and ansible_playbook + ansible_playbook_local12:56
pigmej_local means it's running via ansible ssh thing, non local means it uses our transports12:56
dshulyak_it is better to remove all this _local, i personnaly prefer it, but it is not compliant with all other handlers12:57
pigmejdshulyak_: we need it :(12:57
pigmejto play with node which doesn't have *yet* all stuff that solar needs12:58
pigmejsalmon_: do you remember the exact use case ?12:58
dshulyak_what the difference with plain ssh transport?12:59
pigmejthat you need ansible on remote node12:59
pigmejand puppet12:59
salmon_pigmej: as you wrote, we need to be able to use any node13:00
pigmejfor _local you need ansible without having packages installed13:00
pigmejsalmon_: yeah but it started somehere13:00
dshulyak_yes, thats the difference, but then bash handler should be used13:00
pigmejdshulyak_: yeah but then you code ansible in bash :D13:01
dshulyak_it is not good that there is 2 very different ways13:01
pigmejwe had some discussion about this13:01
pigmejsomewhere is even a document...13:01
pigmejdshulyak_: the problem is that this bash handler would be complicated13:01
pigmejdshulyak_: there you have a doc13:02
pigmejthat's though follow up13:02
salmon_in the end ansible_playbokk remote/local should only stay13:04
pigmejsalmon_: I would even say only 'remote' version13:04
pigmejlocal should be only used internally or "I know what I'm doing mode"13:04
pigmejyou tried to write pure bash version for installing something in poznan13:05
pigmejon early december ;P13:05
salmon_yeah, ansible local is better approach to use ;)13:06
pigmejwell, let's say it another, it's EASIER :D13:06
dshulyak_is there any changes in the workflow to bootstrap solar?13:08
pigmejdshulyak_: "some" would be easiest answer13:08
pigmejyou need new image13:08
dshulyak_can i just install all necessary software?13:08
pigmejshould be enough13:09
pigmejand pip install -e . obviously :)13:09
dshulyak_i wanted to test with sqlite actually13:09
dshulyak_why do i need this overrdie?13:09
pigmejif you want sqlite then you don't need it13:11
pigmejwe switched by default to sqlite because os infra doesn't use riak13:12
pigmejso by default there is sqlite, and we export that override to adjust stuff for vagrant or fuel-devops stuff13:12
dshulyak_hm, then what is changed?13:12
dshulyak_"some" would be easiest answer13:13
salmon_dshulyak_: path to db for sqlite may be wrong13:13
salmon_solar_db: sqlite:////tmp/solar.db13:14
salmon_does it exist ?13:14
pigmejdshulyak_: that's error from docker container?13:14
dshulyak_ah okay13:15
dshulyak_with riak i have more descriptive error13:15
dshulyak_solar.core.resource.repository.ResourceNotFound: Resource definition {'repo': 'templates', 'resource_name': 'nodes', 'version_sign': '>=', 'version': None} not found13:15
pigmejI saw similar in docker or when I used sqlite in and outside riak13:15
pigmejit's not related13:15
pigmejdshulyak_: disk space end ?13:15
dshulyak_so i need to create repositories i think13:15
pigmejwell peewee error was unrelated to this13:16
salmon_wow, so strange error :D13:16
pigmejsalmon_: that ResourceNotFound is ok13:16
pigmejit's because no repositories13:16
pigmejyou need to link13:17
pigmej    - file: src=/vagrant/resources dest=/var/lib/solar/repositories/resources state=link13:17
pigmej    - file: src=/vagrant/templates dest=/var/lib/solar/repositories/templates state=link13:17
pigmejor just execute `solar repository import resources` and then the same for templates13:17
salmon_with link13:17
pigmejeven add -l there13:17
dshulyak_yeah, with sqlite it is still the same and i have a lot of disk space there..13:20
dshulyak_and unit tests are ok13:20
pigmejremove sqlite database then and check again13:22
pigmejbecause it looks like some issue with sqlite db itself (that IO error)13:22
pigmejmaybe it's owned by root?13:22
pigmejdshulyak_: could you review it?13:53
pigmejdshulyak_: your PR is still in progress, right?14:02
dshulyak_pigmej: will do, i still didnt test it with riak strong consistency turned on, but overall i think it will stay this way14:04
pigmejyou mean?14:05
pigmejyou want to have locks with non strong consistent bucket on riak ?14:05
dshulyak_pigmej: yes, the way we are using it currently - one physical node and n_val=1, afair all operations on 1 vnode are sequential and therefore we will always know if lock is acquired by another thread14:12
pigmejdshulyak_: yeah but we're *not* going to recommend one node riak cluster14:13
pigmejriak is cluster system, please don't craete solution which works only on one node14:13
dshulyak_it doesnt matter14:13
pigmejwhole idea of using riak was to have "distributed" stuff14:13
pigmejit matters14:13
dshulyak_how exactly?14:13
pigmejwith distributed env you will not have siblings on write14:13
pigmejor you want ot have n_val = 1 for locking?14:14
dshulyak_yes, we have n_val=1 right now14:14
dshulyak_why i wont have siblings with >1 node ?14:15
pigmejyou will have them but not on write14:15
pigmej(not always)14:15
dshulyak_ah ok, i got it14:15
pigmejI'm not sure though how n_val=1 would behave in distributed env, we will probably voliate all good practices.... ;d14:16
dshulyak_with strongly consistent bucket i expect to have RiakError on write (according to docs), and in _acquire method i am checkin sibglings error both on write and read14:16
pigmejyeah but for read it's uselless14:16
pigmejbecause concurrent operation is already in progress (2)14:16
pigmejif A created lock, and B did it too, then C will notice siblings on read14:17
pigmejbut both A and B will not know about that situation14:17
pigmejso, C will wait but A and B will continue14:17
openstackgerritMerged openstack/solar: Raise better error when requested transport is not found
dshulyak_if A will create lock, and B will create lock with pw=quorum, will i notice siblings on write or on read after this write?14:20
dshulyak_no what?14:20
pigmejyou will not notice it on write14:21
pigmejyou may but you may not14:21
pigmej then go to section named "Strict quorum"14:21
pigmejand check if you meant that or not :)14:22
pigmejI'm not sure what will happen with n_val=1 because I don't know now how this n_val selection works exactly14:22
pigmejbut if I would need to bet, I would say that A and B will succeed, and none of them will know about this situation14:23
pigmejand after some time, riak will create siblings for this.14:24
dshulyak_you are talking about n_val=1 with more than one node in cluster?14:24
pigmejbecause I'm pretty sure that first node which will receive write with n_val will be success14:24
pigmejwith one node riak is probably sequential14:25
pigmejI say "proabably" because I never ever tested this case :)14:25
dshulyak_i would expect that with n_val=1 and pw=1 - one of the concurrent writes will fail, i will try to test today/tommorow, also that part from aphyr blog isnt very clear, what the point of primary vnode if riak will try to preserve write even if primary is not avaiable :) ?14:31
pigmejriak will try to preserve writes because it's how dynamo works14:34
pigmejBUT n_val=1 is interesting case, maybe it will somehow work with that14:34
pigmejthe lowest n_val that I used in production was 314:35
pigmejso.. :)14:35
pigmejdshulyak_: could you in meantime look on my handler patches ? If these are ok for you please merge, then I will continue my last thing at this part :)14:35
dshulyak_also while in MV i started doing some stuff with zeromq based worker14:40
pigmejyeah you mentioned it :)14:42
pigmejI'm not sure if we need zmq but yeah let's see it someday14:44
pigmejwe also need to create that 'small' worker14:44
pigmejwhich will be some inproc stuff but it will be easy probably :)14:44
pigmejdshulyak_: thanks for reviews!14:44
dshulyak_i though about separating scheduler/worker code and communication code, and then for each communication method there will be separate client14:45
pigmejI had similar idea / thoughts14:50
pigmejthen scheduler could be *anything* and worker will be scheduler independent14:51
pigmej(with some limitations obviously)14:51
openstackgerritMerged openstack/solar: Ansible Playbook uses now solar_transports
openstackgerritJedrzej Nowak proposed openstack/solar: Implements ansible_playbook_local handler
pigmejthere was merge conflict dshulyak_ :)15:15
dshulyak_anyone remembers what is the fix for this - mount: unknown filesystem type 'vboxsf' ?15:41
pigmejnever had it15:41
pigmejso I can't help :)15:41
dshulyak_ok, whatever it was update helped15:56
openstackgerritDmitry Shulyak proposed openstack/solar: Preserve original order of keys in
pigmejdshulyak_: sorry for that mistake :D16:41
dshulyak_which mistake?16:43
dshulyak_with order :D ?16:43
pigmejthat I don't preserve order :D16:43
pigmejI coded that part though but I forgot to use it16:44
pigmejI commented PR too, just use list, tuple instead16:44
pigmejwouldn't it be better ?16:44
pigmej(it's already coded)16:44
dshulyak_i used it at first, but then i will iterate again in __init__16:45
salmon_I think, I tested it only with riak16:45
pigmejonly riak uses it16:46
pigmejah no openstack too16:46
pigmejok screw it, it's ok with this ordered dict16:46
pigmejDon't change it dshulyak_ :)16:46
pigmejdshulyak_: anyway, how did you catch it ?16:49
dshulyak_10.0.0.3 on solar-dev2, - solar-dev116:49
pigmejoh interesting16:50
dshulyak_and deployment hanged16:50
pigmejost ?16:50
openstackgerritMerged openstack/solar: Preserve original order of keys in
pigmejI tested ost loles tested it too, ;D16:51
salmon_yeah, interesting16:51
pigmejanwyay, torrent inside docker is bad idea16:51
pigmejit just needs some ports open / exposed, I don't want to hardcode some stuff for our docker container16:51
pigmejcan we instead write in readme that torrent is incompatible with docker now ?16:52
dshulyak_this is same problem with PYTHONHASHSEED16:52
dshulyak_or maybe we can write simple init.d script16:52
pigmejsalmon_: can we test it with devops ?16:53
dshulyak_fuel migrated to centos7, so we dont depend on docker that much16:53
pigmejmaybe we could somehow add some generic post test things that checks something in DB ?16:53
salmon_pigmej: with fuel-devops we are using containers16:53
pigmejdshulyak_: well, if we will get rid of celery + redis...16:53
pigmejsalmon_: I'm talking about 2 things in the same time ;D16:53
pigmejdshulyak_: too :D16:54
dshulyak_i simplified celery start up a lot - celery worker -A solar.orchestration.runner -P gevent -c 1000 -Q system_log,celery,scheduler16:54
pigmejsalmon_: anyway, torrent example works when firt torrent seed is started outside docker16:54
pigmejI don't want to run this docker with privileged mode or something...16:54
pigmejdshulyak_: well the problem is that we have different paths in container and outside16:55
pigmejbut yeah the simpler the better16:58
pigmejdshulyak_: can you do workflow +1 again ? :) It needed rebase :)17:00
pigmejthx :)17:01
openstackgerritMerged openstack/solar: Implements ansible_playbook_local handler
*** dshulyak_ has quit IRC17:39
*** dshulyak_ has joined #openstack-solar17:56
openstackgerritJedrzej Nowak proposed openstack/solar: Fixed torrent transport
pigmejsalmon_: ^ torrent works18:12
pigmejI fixed port range to 6881-6981 and I exposed it in compose18:13
*** dshulyak_ has quit IRC18:48
*** tzn has quit IRC19:01
*** tzn has joined #openstack-solar21:26
-openstackstatus- NOTICE: Gerrit is restarting to resolve java memory issues22:06
openstackgerritMerged openstack/solar: Fixed torrent transport
*** tzn has quit IRC22:40
*** salmon_ has quit IRC23:29

Generated by 2.14.0 by Marius Gedminas - find it at!