[darcs-users] soc progress 3

[Dmitry Kurochkin]

On Thu, Jun 11, 2009 at 1:42 PM, Petr Rockai<me at mornfall.net> wrote:
> Dan Pascu <dan at ag-projects.com> writes:
>
>> Wouldn't a persistent http connection serve better this purpose? Starting with
>> version 1.1 the http protocol supports keep-alive, which  means you only have
>> to make 1 connection and can fetch all the files  over it. I believe that
>> making a lot of one shot connections that only  fetch one file is much more
>> taxing than the fact that there are many  files to transfer. I think that
>> making less connections can result in  a better improvement than reducing the
>> number of files to fetch,  unless that number becomes 1. This is more apparent
>> in the case of  https where the initial connection setup is very expensive.
> We already use pipelining when possible. First, our implementation helps only
> marginally, second only very few systems actually support it.
>
>> In the end both techniques could be applied, it's just my belief that using
>> keep-alive can provide a larger improvement, with less effort.  Later we could
>> see if grouping small files together provides any  significant improvement over
>> that or it's just marginal, in which case  the extra complexity is probably not
>> worth it.
> You could try compiling darcs with curl-pipelining and do some benchmarking. I
> don't know how much this helps with hashed repositories, I just recall this not
> as much as one'd hope.

AFAIK pipelining does not help for hashed repos indeed. Due to the way
darcs downloads patches.

But it really helps for old format repos. Perhaps darcs can be
improved to use pipelining better when getting hashed repos.

Regards,
  Dmitry

>
> Moreover, disk usage is a concern here as well -- most filesystems in real use
> have a block size of some 4 kilobytes.
>
> Yours,
>   Petr.
>
> --
> Peter Rockai | me()mornfall!net | prockai()redhat!com
>  http://blog.mornfall.net | http://web.mornfall.net
>
> "In My Egotistical Opinion, most people's C programs should be
>  indented six feet downward and covered with dirt."
>     -- Blair P. Houghton on the subject of C program indentation
> _______________________________________________
> darcs-users mailing list
> darcs-users at darcs.net
> http://lists.osuosl.org/mailman/listinfo/darcs-users
>