[darcs-users] Re: darcs apply is very, very slow...

Wed Feb 14 12:25:56 UTC 2007

I've tried everything on discussed on the mailing list thread. I added
tags at strategic points in the patch history. I also did a darcs
optimize and darcs optimize --reorder-patches I also did a darcs
optimize --uncompress

Then I tried applying the 190KB patch to the repo, this time I used
`time' and waited it out:

real    349m29.438s
user    288m4.748s
sys     2m59.275s

This is on a workstation with P4 1.7GHz CPU and 1GB of memory.

To recap the repository contains only 141 files. The patch although a
bit larger than what I usually apply to this tree is pretty small at
190KB and touches 127 of those files.

Based on the time results above, 5 hours to apply a patch this small
to a repo of this size is a bit too much for me.

If anyone else would be willing to try it out or if anyone in the
development team is interested in looking at the problem, here is the
repository:

http://abing.gotdns.com/darcs/repos/cake/

Here is the patch:

http://abing.gotdns.com/cake.dcs

Do a darcs get from the repository first and then when you have your
local copy, download the patch and do a darcs apply to your local
copy.

Incidentally, when I ran the above timed run of darcs I used darcs
apply --verbose

The process seems to stop at the stage where it is "diffing dirs". I
was not able to copy the whole verbose output as it scrolled through
the terminal.

Here is my darcs --exact-version:

nimrod at iwojima:$ darcs --exact-version
darcs compiled on Feb 11 2007, at 16:10:46
# configured Fri Jun 16 14:55:21 EDT 2006
./configure --no-create --no-recursion

Context:

[TAG 1.0.8
Tommy Pettersson <ptp at lysator.liu.se>**20060616160213]

I used the debian build scripts to build my own .deb with
--enable-antimemoize option for ./configure.

On 2/13/07, Nimrod A. Abing <nimrod.abing at gmail.com> wrote:
> Hello,
>
> I have found this thread started only some days ago:
>
> http://lists.osuosl.org/pipermail/darcs-users/2007-February/010830.html
>
> I'm going to try to follow what's been done on this thread to see if
> it works for me as well. The size of the repos discussed there have
> run into hundreds of MBs. My repo is only 3MB and yet darcs push/apply
> is very slow. Here is some more info on the repos I have:
>
> * Both repos are on the same filesystem on the same computer. I think
> this is a very important detail i neglected to mention in my last
> post.
> * Both repos use tags.
> * The main repo has some patches that are not in the work repo. The
> patches in the work repo may sometimes introduce conflicts.
>
> Anyway, I will try the suggestions posted in the thread I found above
> and post back what I come up with here in the next few days.
>
> On 2/11/07, Nimrod A. Abing <nimrod.abing at gmail.com> wrote:
> > Hello,
> >
> > I have a repository consisting of 141 files. I maintain two copies of
> > this repo, one for recording changesets for public consumption and the
> > other being my working repository.
> >
> > Recently 127 of those files needed changes to their copyright info. I
> > made those changes in my working repo and then when time came to push
> > to the public repo, darcs just sits there. I left it running for about
> > a hour and when I came back, it still had not finished. I don't think
> > that it crashed because darcs soaks up 85% of my CPU. Finally I gave
> > up after about 30 minutes (total running time about 1 hour 30
> > minutes).
> >
> > The patch is about 190K with 3,654 lines of text and touches 127 of
> > those 141 files. The changes to each file are small. About 4-6 lines
> > on average. Although some files have about 10-15 lines modified and
> > possibly have some conflicts as well.
> >
> > This is the first time that I have applied a changeset this large. I
> > figured size would not be an issue since it only touches 127 files. I
> > have also tried rebuilding darcs using the --enable-antimemoize option
> > and tried re-applying the same patch with the new version of darcs.
> > There were no noticeable improvements whatsoever.
> >
> > I then tried something else. I broke up the patch into several
> > subpatches. Using some shell scripting I had darcs create one patch
> > for each file that was changed.
> >
> > This time I tried pushing each individual patch. This time I also
> > timed the operation using the time command. Here are some results:
> >
> > [~/Temp/cake-dev]
> > nimrod at iwojima:$ time darcs push --patch='Merge changes from
> > CakePHP-v-1.1.13.4450 PART 0000000004' -a
> > Pushing to "/home/virtual/cake-dev/html/cake"...
> > Finished applying...
> >
> >
> > real    0m57.239s
> > user    0m47.367s
> > sys     0m0.748s
> > [~/Temp/cake-dev]
> > nimrod at iwojima:$ time darcs push --patch='Merge changes from
> > CakePHP-v-1.1.13.4450 PART 0000000005' -a
> > Pushing to "/home/virtual/cake-dev/html/cake"...
> > Finished applying...
> >
> >
> > real    0m58.420s
> > user    0m47.223s
> > sys     0m0.768s
> > [~/Temp/cake-dev]
> > nimrod at iwojima:$ time darcs push --patch='Merge changes from
> > CakePHP-v-1.1.13.4450 PART 0000000006' -a
> > Pushing to "/home/virtual/cake-dev/html/cake"...
> > Finished applying...
> >
> >
> > real    0m53.750s
> > user    0m46.779s
> > sys     0m0.716s
> >
> > Each patch is about 1K in size now. And after seeing the results
> > above, I'm a bit disappointed that it takes this long to apply a
> > single patch. There are 127 of these smaller patches and each one
> > takes about a minute to apply. By these figures it will take over 2
> > hours to apply all these patches.
> >
> > Is there anyway to improve this? I am willing to subject myself to
> > experimentation :) since this repo is only my side-project and I keep
> > backups of everything.
> >
> > Thanks in advance.
> > --
> > _nimrod_a_abing_
> >
> > [?] http://abing.gotdns.com
> >
>
>
> --
> _nimrod_a_abing_
>
> [?] http://abing.gotdns.com
>

-- 
_nimrod_a_abing_

[?] http://abing.gotdns.com