Friday, 2016-03-04

*** angdraug has quit IRC00:40
*** openstack has joined #openstack-solar04:29
*** openstack has quit IRC04:30
*** openstack has joined #openstack-solar04:32
*** openstack has quit IRC04:34
*** openstack has joined #openstack-solar04:36
*** openstack has quit IRC04:36
*** openstack has joined #openstack-solar04:37
*** openstack has quit IRC04:39
*** openstack has joined #openstack-solar14:07
*** openstackgerrit_ has quit IRC14:36
*** openstackgerrit_ has joined #openstack-solar14:37
dshulyaksolar becomes broken if i accidentally will try to create same node 2 times :)14:45
dshulyaksolar.dblayer.solar_models.DBLayerSolarException: No input port for ssh_transport_node_414:45
pigmejthere is a chance14:45
pigmejbecause we don't check for duplicates at all14:46
pigmejor for cycles in connections afai14:46
dshulyakdo we have transaction rollback on error for sql backends ?14:46
pigmejdshulyak: even if don't implement it14:48
dshulyakit seems as something fails in the middle, and we end up with crappy state14:48
pigmejit could be implemented but don't doo it14:48
salmon_we need a solution for duplicates, it's annoying14:48
pigmejbecause you can't support the same for riak14:48
salmon_and i would say we need it asap :P14:49
pigmejjust don't do rollback on error, then we should drop riak completely14:50
pigmejand redesign dblayer for "relational" stuff14:51
pigmejwe could write simple resolver which will revert previous version14:52
pigmejbut still, there is no way of creating let's say decent and stable "create if not exists"14:52
salmon_we just need to check for duplicates :P14:52
dshulyaki think that we need transcations14:52
pigmejsalmon_: whan if 2 funcitons do the same check in the same time?14:52
salmon_pigmej: we have locks already, we may have one more :P14:53
pigmejsalmon_: and performance-- like 100s times14:53
salmon_at least it will be usable14:53
salmon_when  you create 2 nodes with the same name it's unusable14:54
dshulyakyes, it is no go to use solar res clear_all  in prod:)14:54
pigmejwell, the duplicate resource is another story14:54
pigmejanyway why do you create the same resource twice?14:55
dshulyakit is wrong question14:55
dshulyaku want user to check in solar if something exists before every operation?14:55
salmon_pigmej: because solar lets me to do it :)14:55
pigmejok we can change this by stroing used names "somewhere"14:57
pigmejin riak in strong consistent bucket (then we can easily cover in code inconsistent state)14:57
pigmejand also you will be unable to create the same resource twice14:58
dshulyakalso it seems to me that such operations as solar create resource should be atomic, otherwise if it will fail 1st time in the middle - we will have inconsistent state in database14:58
dshulyakand every 2nd operation will lead to error14:58
dshulyakevery next14:58
pigmejsingle resource.create is atomic14:59
dshulyakwith inputs?14:59
pigmejinputs are stored inside resource14:59
pigmejso yes, with inputs15:00
dshulyaki meant with connections15:00
pigmejconnections are given on resource.connect()15:00
pigmejand it's also atomic, because you resoruce.connect(other_resouce)15:00
dshulyakyes, but if u do creation of vr - it is not atomic :)15:01
pigmejyou mean several resources?15:01
dshulyakfrom user point of view - vr is single operation, isnt it?15:01
pigmejseveral resources == several db objects != atomic15:02
dshulyakyeah, but using transcations it could be15:02
dshulyakand i think thats the problem for us15:02
pigmejthen mv riak Trash/15:02
dshulyakor implement our own transcations stuff, yes? which is not very good idea15:03
pigmejit's possible but for sure not now :)15:03
pigmejwe could do some post check on riak though15:03
pigmejalso, I'm not sure if you need transactions for this, what we for sure need is to *not* fuckup database15:04
pigmejlike we can do now15:04
dshulyakbtw resource.connect cretes 1 or atleast 2 indexes? one on receiver, another on emitter?15:04
pigmejonly receiver15:04
dshulyakso only 1?15:04
pigmejor emitter because I don't remebmer exacly15:04
pigmejseveral indexes but bound to single object15:04
pigmejand all operations inside single objects are "all or none"15:05
pigmejit seems that connections are stored in emitter15:09
pigmej(I always mess it up so I may be wrong :D)15:09
pigmejsalmon_: what would you expect on name conflict ?15:21
pigmejI can fix it in ~1minute15:21
pigmejit will be ugly fix15:21
pigmejbut it will allow you to work15:21
salmon_pigmej: exception will be fine15:22
pigmejI can do naive check15:22
pigmejwith big warning in code15:22
openstackgerritBogdan Dobrelya proposed openstack/solar: Separate vagrantfile for the docker provider
pigmejthat it will work only under certain conditions15:22
salmon_what conditions?15:25
pigmejno concurrent env15:26
pigmejmaybe later we could add something smarter15:26
pigmejlike write smarter conflict solver there15:26
pigmejwhich could inform you that something is fucked up, and it should provide then way to solve this15:26
dshulyakthis anchors are really painfull.. we need somehow insert them every time something needs is changed/added15:29
pigmejAnd I still don't know why that graph part is slow15:31
pigmejlazy_save is pretty much everywhere ;/15:31
openstackgerritJedrzej Nowak proposed openstack/solar: Added naive method to check if object already exists
pigmejsalmon_: with dedication for you :)16:15
pigmejdshulyak: after we will land F2S we can sit and fix our things for "already exists", sounds like a plan?16:16
pigmejsalmon_: consider this as "hacky" solution ;)16:21
pigmejwe may add later smarter way for each backend16:21
pigmejok I know why PG is 100% cpu16:58
pigmejbecause of broken pool implementation, it's too complicated and it causes ~80% of load16:58
pigmejit seems...16:58
pigmejand it's slow16:58
pigmejI will swap it and it will be probably fast agai16:58
*** angdraug has joined #openstack-solar17:00
pigmejbut that pool is not used by default, so on non pool stuff, it's even worse, and there is our reason :)17:01
pigmejI wasted day for it;/17:01
pigmejok like half17:01
pigmejok even with default pool it's better, but that pool is stupid as hell, on monday I will write custom pool that will suit our needs17:24
pigmejon pg: Total Delta: 8.56165814417:25
pigmejTotal Time: 8.5665950775117:25
pigmejand no 100%17:25
pigmejdshulyak: salmon_17:25
salmon_I use sqlite but cool ;)17:26
pigmejdo you have 100% cpu ?17:26
pigmejwith this and solar o run-once last (stage && process before) on PG you will have 100% CPU17:27
salmon_I have17:27
pigmejon sqlite?17:28
pigmejfor sqlite I even can understand 100% without optimization in that garphs update17:28
*** dshulyak has left #openstack-solar17:51
*** salmon_ has quit IRC22:53
*** angdraug has quit IRC23:16

Generated by 2.14.0 by Marius Gedminas - find it at!