Brad Fitzpatrick (brad) wrote,
Brad Fitzpatrick
brad

smart migrator

I just wrote a database migration script which, after each chunk of data moved, asks the load balancer (Perlbal) the free user queue depth. If more than 10 (less is just noise), it sleeps a second and asks again. Only once the queues are empty does it migrate more data.

End result: it moves data as fast as it can, without affecting page response times.

1432000/81000000 = 1.77%
1433000/81000000 = 1.77%
1434000/81000000 = 1.77%
wait_back = 10
1435000/81000000 = 1.77%
1436000/81000000 = 1.77%
....
1450000/81000000 = 1.79%
1451000/81000000 = 1.79%
1452000/81000000 = 1.79%
wait_back = 35
1453000/81000000 = 1.79%
1454000/81000000 = 1.80%
1455000/81000000 = 1.80%
...
1470000/81000000 = 1.81%
1471000/81000000 = 1.82%
1472000/81000000 = 1.82%
wait_back = 14
wait_back = 10
wait_back = 67
wait_back = 68
wait_back = 50
wait_back = 67
wait_back = 91
wait_back = 72
1473000/81000000 = 1.82%
1474000/81000000 = 1.82%
1475000/81000000 = 1.82%

I've been meaning to make a generic wrapper for any utility, where the wrapper parent watches the load balancers, and the child does its work at full speed, but the parent will occasionally SIGSTOP/SIGCONT it...... haven't got around to that yet. The generic wrapper could even have another pluggable-child as its rate-limit determiner, so anybody could use it.
Subscribe
  • Post a new comment

    Error

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
  • 10 comments