[tor-bugs] #25688 [Obfuscation/Snowflake]: proxy-go is still deadlocking occasionally

[Tor Bug Tracker & Wiki]

#25688: proxy-go is still deadlocking occasionally
-----------------------------------+------------------------
 Reporter:  dcf                    |          Owner:  (none)
     Type:  defect                 |         Status:  new
 Priority:  Low                    |      Milestone:
Component:  Obfuscation/Snowflake  |        Version:
 Severity:  Normal                 |     Resolution:
 Keywords:                         |  Actual Points:
Parent ID:                         |         Points:
 Reviewer:                         |        Sponsor:
-----------------------------------+------------------------

Comment (by dcf):

 Replying to [ticket:25688 dcf]:
 > In the meantime, I've put the fallback proxies back on periodic
 restarts. Before the intervals were 1h,2h,10h; now I increased them to
 17h,23h,29h (prime numbers, so the average time before the next restart is
 < 17h).
 >
 > I'll update this ticket with a graph showing uptimes when I have time.

 This graph shows restarts ╳, as well as polls
 [[span(style=color:lightgray,•)]] and data transfer
 [[span(style=color:orange,•)]], so you can see where they got stuck after
 3 or 4 days.

 The rows '''1h''', '''2h''', '''10h''', '''17h''', '''23h''', '''29h'''
 are set to periodically restart with no memory limit. The rows '''a''',
 '''b''', '''c''' ran with different memory limits at different times:
 ||2018-03-22 03:19:19 ||all `ulimit -v` 400 MB ||
 ||2018-03-22 05:16:59 ||'''a''' appengine unlimited, '''c''' appengine
 `ulimit -v` 800 MB, all others `ulimit -v` 400 MB ||
 ||2018-03-23 01:02:18 ||'''a''' appengine unlimited, all others `ulimit
 -m` 200 MB ||
 ||2018-03-27 19:26:12 ||'''a''', '''b''', '''c''' unlimited ||
 `ulimit -v` caused quick deadlocks; `ulimit -m` was less quick but it
 still happened, and it still happened even with no `ulimit`.

 Scroll right →→→
 [[Image(proxy-go-starting.20180403.png)]]

 Memory use plateaus around the time of the hangs, though that may be as a
 result rather than a cause. The server has 2 GB and it doesn't get close
 to that. (Beware the horizontal axis doesn't quite line up with the graph
 above.)
 [[Image(proxy-go-mem.20180403.png)]]

 File descriptors seem stable.
 [[Image(proxy-go-fd.20180403.png)]]

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25688#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online