Brad Fitzpatrick (brad) wrote,
Brad Fitzpatrick
brad

Pattern sought

I often write code like:

foo();
... (no branches) ...
bar();

Where foo() and bar() both fetch their own data to work on first, usually over the network with a fair bit of latency.

If the compiler could tell foo() and bar() will both be executed, since there are no branches in between, it'd be nice if the compiler could put some code in parallel for me, and merging the data requests, eliminating the unnecessary round-trip latency.

I want to work with a language that allows me to give the compiler lots of information about what it can rearrange, and how.

Lacking that, I'm searching for a design pattern to make the above moderately easy to write and yet still be understandable to others. (which are largely the same goal... clean code is easier for everybody)

Something like making foo and bar return tuples of the data they will need, and a delegate to finish their operation, once they have the data. But any way I think about it doesn't end up working. I really think I need a language with lazy evaluation.

If I do:

a = foo();
....
b = bar();
...

The lazy language could do nothing yet, since I haven't referenced a or b, and foo/bar have no side-effects. But then later I do:

force_eval(a, b); // merges the over-the-network data retrieval

Before later code which uses a and b.

Am I just dreaming? I know there are lazy languages out there, but are they actually used, or they just academic wanking?

Barring that, what about this:

preloaded_data = find_data_to_preload()
a = foo(preloaded_data);
...
b = bar(preloaded_data);

Where find_data_to_preload() walks the op tree from the callers position, asking subroutines along the way if they'll be wanting any data, which they'll evaluate dynamically based on the current scoping. The walker will stop once it hits a branch which is based on something it doesn't know yet.

For instance:

journal = LJ::load_user("photography"); # a community;
preload = find_data_to_preload()
metadata = LJ::load_account_settings(preload, journal);
if (is_community(journal)) {
metadata += LJ::load_community_settings(preload, journal)
}

if (metadata[...] .....) {

}

So find_data_to_preload() at that point would walk along, ask load_account_settings if it supported the "wants preloaded data" interface, then it'd give it a temporary or read-only copy of the current scope. load_account_settings would say, "I'll need the following data....". Then the walker keeps going. It evaluates the expression "is_community(journal)" which it knows has no side-effects (either because of static analysis or metadata or both) and keeps walking inside, asking "load_community_settings" if it wants data preloaded.

Once it gets to the second branch, "if (metadata[..])" it stops because it hasn't loaded the metadata for that journal yet.

This seems perfectly possible to implement in either Perl or .NET, where you can walk the op tree. Probably painful, though.

Other suggestions/patterns?
Tags: perl, tech
Subscribe
  • Post a new comment

    Error

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
  • 14 comments