svn2git:mnaranjo-svn2git.git
7 years agoAdd support for default values in variable substitutions.
Nicolás Alvarez [Mon, 11 Apr 2011 21:43:40 +0000 (18:43 -0300)]
Add support for default values in variable substitutions.

If there is no variable 'foo' set, ${foo} exits with a fatal error.
With this patch, you can use ${foo|sometext}, which will substitute to
'sometext' if the variable 'foo' isn't set. The default text may be empty.

7 years agoLimit variable names to letters, numbers and underscores.
Nicolás Alvarez [Mon, 11 Apr 2011 21:36:37 +0000 (18:36 -0300)]
Limit variable names to letters, numbers and underscores.

The parser used to allow any non-whitespace character, which could
cause problems in practice. For example, you could create a variable
with } or = in the name, or a character we may want to use in a syntax
extension later.

7 years agoSave deleted branches in a visible namespace 1.0.4
Sebastian Pipping [Fri, 8 Apr 2011 05:16:44 +0000 (07:16 +0200)]
Save deleted branches in a visible namespace

 - The branch is first created properly and then deleted
   (line "progress SVN r77 branch config-header = :0 # delete")

 - Jehan's post-1.0.3 commit 584005f2e26149282aa12c4a0367d250caaf3918
   writes a backup of the branches latest ref to refs/backup/
   on deletion.  Many thanks for that commit!

However, in my opinion the problem with the current code is that it
makes it much too easy to lose history during the conversion to Git.
Without a tag or branch pointing to it the commits will fall off easily,
especially if people use "git clone" on the bare Git repository created
by svn2git.

It would be safer and useful to create visible tags for this, e.g.
refs/tags/branch-bugfix17-deleted.

My attached patch changes Jehan's code to do just that.
Please consider application - this patch really matters to me.
If you'd rather have both a ref in refs/backup/ and in refs/tags/ or
require a command line option for it I can adjust the patch for you.

Thanks for listening,

7 years agoCreate backup reference before deletion of the original branch
Jehan Bing [Thu, 10 Mar 2011 18:54:52 +0000 (10:54 -0800)]
Create backup reference before deletion of the original branch

Ensure that the backup reference is created before deleting the original
branch. As it was, the deletion was executed earlier causing the backup
reference to point to a non-existing branch so was not created.

7 years agoRemove unnecessary const casts.
Nicolás Alvarez [Wed, 16 Mar 2011 20:37:22 +0000 (17:37 -0300)]
Remove unnecessary const casts.

7 years agoCommit branch creation when requested and not on next transaction commit 1.0.3
Torgny Nyblom [Mon, 7 Feb 2011 19:37:39 +0000 (20:37 +0100)]
Commit branch creation when requested and not on next transaction commit

Branches was created by adding the command to a list, this list was then
written to git when the next transaction was committed. This had the
side effect that if the branch creation was the last thing the branch
was never created.

7 years agoFix encoding of author names.
Nicolás Alvarez [Sat, 29 Jan 2011 20:38:34 +0000 (17:38 -0300)]
Fix encoding of author names.

Author names were being properly passed through fromUtf8(),
but then sent to a QTextStream using the system's locale.
This patch forces the text stream to always use UTF-8.

7 years agoSupport ranges in the revisions file
Torgny Nyblom [Mon, 17 Jan 2011 09:30:21 +0000 (10:30 +0100)]
Support ranges in the revisions file

7 years agoDelay transaction creation untill usage.
Torgny Nyblom [Sun, 16 Jan 2011 05:55:43 +0000 (06:55 +0100)]
Delay transaction creation untill usage.

This might fix the issue with an extra empty diff commit before all
tags.

7 years agoPrint how we were invoked
Torgny Nyblom [Sat, 15 Jan 2011 19:51:59 +0000 (20:51 +0100)]
Print how we were invoked

7 years agoOnly print warning if it applies
Torgny Nyblom [Sat, 15 Jan 2011 18:49:42 +0000 (19:49 +0100)]
Only print warning if it applies

7 years agoFix serious performance regression.
Nicolás Alvarez [Thu, 23 Dec 2010 20:28:52 +0000 (17:28 -0300)]
Fix serious performance regression.

In Repository::commit, don't call startFastImport() if we have nothing to
write to the fastImport stream. startFastImport() may start new
git-fast-import processes if they were previously killed, so it may be
extremely slow to call it frequently if it's not necessary.

7 years agoAdd option for using the real content of an SVN branch when creating a 1.0.2
Torgny Nyblom [Tue, 21 Dec 2010 13:49:54 +0000 (14:49 +0100)]
Add option for using the real content of an SVN branch when creating a
new branch instead of using the contents of the git "from" branch.

7 years agoFix the --version option 1.0.1
Torgny Nyblom [Tue, 21 Dec 2010 09:30:15 +0000 (10:30 +0100)]
Fix the --version option

When supplying --version the git commit that was used to create the
version should be printed to stdout.

7 years agoMake sure that there are merges recorded before trying to read the last one
Torgny Nyblom [Thu, 16 Dec 2010 10:34:39 +0000 (11:34 +0100)]
Make sure that there are merges recorded before trying to read the last one

7 years agoTry and fix cvs2svn multiple merge points for branches and tags.
Torgny Nyblom [Thu, 16 Dec 2010 08:52:10 +0000 (09:52 +0100)]
Try and fix cvs2svn multiple merge points for branches and tags.

Might still need some logic for detecting the correct from branch.

7 years agoMake the tagging commands visible in the gitlog aswell
Torgny Nyblom [Mon, 20 Dec 2010 20:12:27 +0000 (21:12 +0100)]
Make the tagging commands visible in the gitlog aswell

7 years agoOnly fetch revision properties once per revision
Torgny Nyblom [Thu, 16 Dec 2010 08:50:31 +0000 (09:50 +0100)]
Only fetch revision properties once per revision

7 years agoOnly call startFastImport when needed
Torgny Nyblom [Thu, 16 Dec 2010 08:49:51 +0000 (09:49 +0100)]
Only call startFastImport when needed

7 years agoBe more clear about what are warnings
Torgny Nyblom [Thu, 16 Dec 2010 08:49:04 +0000 (09:49 +0100)]
Be more clear about what are warnings

7 years agoMove branch creation/deletion/restting to Repository and write these in
Torgny Nyblom [Wed, 15 Dec 2010 15:03:34 +0000 (16:03 +0100)]
Move branch creation/deletion/restting to Repository and write these in
commit()

Prepare for handling cvs2svn borked tags/branches

7 years agoDitch PrefixingRepository in favor of a memmber variable in
Torgny Nyblom [Wed, 15 Dec 2010 14:29:47 +0000 (15:29 +0100)]
Ditch PrefixingRepository in favor of a memmber variable in
[FastExport]Repository
Some minor reordering

7 years agoDelete before other changes 1.0.0
Torgny Nyblom [Mon, 13 Dec 2010 17:11:22 +0000 (18:11 +0100)]
Delete before other changes

Fix issue where if a branch reset was triggered before a branch deletion
in the same revision the reset was overridden by the deletion

7 years agocall startFastImport where it is used.
Torgny Nyblom [Mon, 13 Dec 2010 07:09:12 +0000 (08:09 +0100)]
call startFastImport where it is used.

7 years agoCode style
Torgny Nyblom [Mon, 13 Dec 2010 07:07:36 +0000 (08:07 +0100)]
Code style

7 years agoDuh, do not try and extract options before they are parsed.
Torgny Nyblom [Sun, 12 Dec 2010 11:32:16 +0000 (12:32 +0100)]
Duh, do not try and extract options before they are parsed.

7 years agoAdd an option to print some stats after a run.
Torgny Nyblom [Sat, 4 Dec 2010 19:53:43 +0000 (20:53 +0100)]
Add an option to print some stats after a run.

7 years agoAllow more then one variable to be used on a line.
Torgny Nyblom [Tue, 30 Nov 2010 09:37:55 +0000 (10:37 +0100)]
Allow more then one variable to be used on a line.

7 years agoQList<int> -> QSet<int>
Torgny Nyblom [Sat, 27 Nov 2010 06:04:52 +0000 (07:04 +0100)]
QList<int> -> QSet<int>

7 years agoAdd an option to parse a list (in a file) of revisions that should be
Torgny Nyblom [Fri, 26 Nov 2010 11:40:15 +0000 (12:40 +0100)]
Add an option to parse a list (in a file) of revisions that should be
used.

7 years agoMerge branch 'master' of gitorious.org:svn2git/svn2git
Torgny Nyblom [Sun, 14 Nov 2010 09:34:30 +0000 (10:34 +0100)]
Merge branch 'master' of gitorious.org:svn2git/svn2git

7 years agoFix detecting directory moves
Niko Sams [Sat, 13 Nov 2010 13:33:26 +0000 (14:33 +0100)]
Fix detecting directory moves

is_dir wasn't reported correctly, I guess because the path didn't exist anymore
at that revision.
Reuse the existing wasDir function as it works perfectly.

With not detecting the path as a dir, the / got not added, an thus the rule for the source
not found.

7 years agoPrint revision information with "copy from" warning
Torgny Nyblom [Mon, 1 Nov 2010 16:12:27 +0000 (17:12 +0100)]
Print revision information with "copy from" warning

7 years agoFix issue with variables not being defined when reading include files
Torgny Nyblom [Mon, 1 Nov 2010 09:49:08 +0000 (10:49 +0100)]
Fix issue with variables not being defined when reading include files

7 years agoPrint the missing from path when warning about missing source
Torgny Nyblom [Mon, 1 Nov 2010 09:40:45 +0000 (10:40 +0100)]
Print the missing from path when warning about missing source

7 years agoFix filename and linenumber for included rulefiles.
Torgny Nyblom [Tue, 26 Oct 2010 10:17:40 +0000 (12:17 +0200)]
Fix filename and linenumber for included rulefiles.

7 years agoBetter include path (FreeBSD patch from Uli - http://gitorious.org/~uqs)
Torgny Nyblom [Mon, 18 Oct 2010 05:54:51 +0000 (07:54 +0200)]
Better include path (FreeBSD patch from Uli - gitorious.org/~uqs)

7 years agoAllow files to be matched
Torgny Nyblom [Fri, 8 Oct 2010 17:25:49 +0000 (19:25 +0200)]
Allow files to be matched

7 years agoMake sure fastImport is started before adding file
Niko Sams [Thu, 7 Oct 2010 18:49:59 +0000 (20:49 +0200)]
Make sure fastImport is started before adding file

7 years agoMerge branch 'master' of gitorious.org:svn2git/svn2git
Niko Sams [Mon, 4 Oct 2010 18:46:53 +0000 (20:46 +0200)]
Merge branch 'master' of gitorious.org:svn2git/svn2git

7 years agosupport svn commit that converts a link into a file in one commit
Niko Sams [Mon, 4 Oct 2010 18:44:13 +0000 (20:44 +0200)]
support svn commit that converts a link into a file in one commit

this happens in kde svn rev 841619

7 years agoAllow more then one rule file to be used in a single run.
Torgny Nyblom [Wed, 29 Sep 2010 17:53:24 +0000 (19:53 +0200)]
Allow more then one rule file to be used in a single run.

7 years agoconst++
Torgny Nyblom [Wed, 29 Sep 2010 17:46:09 +0000 (19:46 +0200)]
const++

7 years agoDie when a rule file contains an invalid regexp
Torgny Nyblom [Wed, 29 Sep 2010 17:45:20 +0000 (19:45 +0200)]
Die when a rule file contains an invalid regexp

7 years agoUnify debug messages and Match structs
Torgny Nyblom [Tue, 28 Sep 2010 18:40:05 +0000 (20:40 +0200)]
Unify debug messages and Match structs

7 years agocall startFastImport() in commit()
Niko Sams [Fri, 24 Sep 2010 19:44:29 +0000 (21:44 +0200)]
call startFastImport() in commit()

This fixes importing large svn commits that get imported into a large number
of git repositories. It happened that a process was closed (in ProcessCache::touch)
before commit() and so the commit wasn't imported correctly.

And remove the touch() call as that is done now in startFastImport()

7 years agotouch fastImport process in startFastImport()
Niko Sams [Fri, 24 Sep 2010 19:39:22 +0000 (21:39 +0200)]
touch fastImport process in startFastImport()

This is needed to make sure that not too many processes are running and
we run out of ressources.
Calling touch only in commit is not enough as startFastImport is called in other
functions as createBranch too - and that can result in too many processes.

7 years agoadd assertions to write() methods to make sure nothing is written into a closed fast...
Niko Sams [Fri, 24 Sep 2010 19:15:32 +0000 (21:15 +0200)]
add assertions to write() methods to make sure nothing is written into a closed fast-import

7 years agooutput repository name that crashed, helpful to know which logs to look at
Niko Sams [Fri, 24 Sep 2010 19:13:44 +0000 (21:13 +0200)]
output repository name that crashed, helpful to know which logs to look at

7 years agoReadd br.marks.last() to various places as a mark of "0" is used to mark
Torgny Nyblom [Fri, 17 Sep 2010 11:19:37 +0000 (13:19 +0200)]
Readd br.marks.last() to various places as a mark of "0" is used to mark
a branch as deleted.

Thanks Raja R Harinath for spotting it.

7 years agoFix assert when br.marks is empty
Torgny Nyblom [Wed, 15 Sep 2010 04:52:02 +0000 (06:52 +0200)]
Fix assert when br.marks is empty

7 years agoMake the fastImport into a logging instance, logging is enabled when the "--debug...
Torgny Nyblom [Mon, 13 Sep 2010 10:19:04 +0000 (12:19 +0200)]
Make the fastImport into a logging instance, logging is enabled when the "--debug-rules" flag is active.

7 years agoAdd posibility to use variables in rule files
Torgny Nyblom [Fri, 10 Sep 2010 10:30:12 +0000 (12:30 +0200)]
Add posibility to use variables in rule files
"declare var=value"
now "${var}" in any line will be replaced by "value"

7 years agoSpaces no tabs
Torgny Nyblom [Thu, 9 Sep 2010 10:08:40 +0000 (12:08 +0200)]
Spaces no tabs

7 years agoMake it possible to use "include file" for including the rules in "file".
Torgny Nyblom [Thu, 9 Sep 2010 10:05:08 +0000 (12:05 +0200)]
Make it possible to use "include file" for including the rules in "file".

7 years agoBetter log messages
Torgny Nyblom [Thu, 9 Sep 2010 08:02:21 +0000 (10:02 +0200)]
Better log messages

7 years agoSpaces not tabs
Torgny Nyblom [Wed, 8 Sep 2010 19:47:54 +0000 (21:47 +0200)]
Spaces not tabs

7 years agoOnly add a '/' if the path is a dir
Torgny Nyblom [Wed, 8 Sep 2010 19:38:06 +0000 (21:38 +0200)]
Only add a '/' if the path is a dir

7 years agoMore verbose debug output
Torgny Nyblom [Wed, 8 Sep 2010 19:36:45 +0000 (21:36 +0200)]
More verbose debug output

7 years agoInter branch merging causes git log to break with fatal internal errors
Torgny Nyblom [Wed, 8 Sep 2010 19:35:19 +0000 (21:35 +0200)]
Inter branch merging causes git log to break with fatal internal errors

7 years agoDon't print the same message as in Transaction()
Torgny Nyblom [Wed, 8 Sep 2010 19:34:15 +0000 (21:34 +0200)]
Don't print the same message as in Transaction()
Creation is already printed at the bottom.

7 years agoTry and work around cvs2svn branching/tagging issues where the directory tree is...
Torgny Nyblom [Wed, 8 Sep 2010 19:33:07 +0000 (21:33 +0200)]
Try and work around cvs2svn branching/tagging issues where the directory tree is not what it seems like

7 years agoMake sure that the path part is empty before deleting a whole branch. If the prefix...
Torgny Nyblom [Sat, 4 Sep 2010 06:27:05 +0000 (08:27 +0200)]
Make sure that the path part is empty before deleting a whole branch. If the prefix option for a rule is used current == svnprefix does not mean that we are dealing with the root of a branch, path needs to be empty as well.

7 years agoWhite space cleanup & debug statments
Torgny Nyblom [Sat, 4 Sep 2010 06:25:45 +0000 (08:25 +0200)]
White space cleanup & debug statments

7 years agoFix character encoding
Torgny Nyblom [Wed, 25 Aug 2010 09:01:11 +0000 (11:01 +0200)]
Fix character encoding

7 years agoMerge branch 'master' of git://gitorious.org/svn2git/svn2git
Raja R Harinath [Mon, 23 Aug 2010 18:19:54 +0000 (23:49 +0530)]
Merge branch 'master' of git://gitorious.org/svn2git/svn2git

The conflict was mainly around two different approaches to fixing the linear
commitMarks issue.  The precise tracking of commit marks per branch is better,
and that's how I resolved the merge conflict.

While at it, I also removed the "revision-*" files, since the "log-*" files
already have the same information, and the branch information too.

Conflicts:
src/repository.cpp
src/repository.h

7 years agoMerge commit 'refs/merge-requests/7' of git://gitorious.org/svn2git/svn2git into...
Torgny Nyblom [Wed, 18 Aug 2010 10:54:03 +0000 (12:54 +0200)]
Merge commit 'refs/merge-requests/7' of git://gitorious.org/svn2git/svn2git into integration

7 years agoMerge commit 'refs/merge-requests/5' of git://gitorious.org/svn2git/svn2git into...
Torgny Nyblom [Wed, 18 Aug 2010 10:41:23 +0000 (12:41 +0200)]
Merge commit 'refs/merge-requests/5' of git://gitorious.org/svn2git/svn2git into integration

7 years agoMerge commit 'refs/merge-requests/1' of git://gitorious.org/~marcguenther/svn2git...
Torgny Nyblom [Mon, 9 Aug 2010 12:24:37 +0000 (14:24 +0200)]
Merge commit 'refs/merge-requests/1' of git://gitorious.org/~marcguenther/svn2git/marcguenther-svn2git into integration

7 years agoExit if unable to open repository
Torgny Nyblom [Mon, 9 Aug 2010 11:32:14 +0000 (13:32 +0200)]
Exit if unable to open repository

7 years agoIf a branch is created from a previous commit that doesn't touch the branch from...
Torgny Nyblom [Mon, 9 Aug 2010 11:31:49 +0000 (13:31 +0200)]
If a branch is created from a previous commit that doesn't touch the branch from path then fallback to using the latest version and issue a warning

7 years agoProvide a way to merge repositories
Raja R Harinath [Thu, 22 Jul 2010 13:02:21 +0000 (18:32 +0530)]
Provide a way to merge repositories

Suppose you have multiple repositories in SVN that you want to merge into
a single one in GIT, it can get very messy to handle all the special-case
rules.

Instead, we introduce a new "forwarding repository" concept, which looks like

  repository subordinate
repository unified
prefix foo/
  end repository

This forwards all commits on the "subordinate" SVN tree to the "unified" GIT
tree, with each file prefixed with "foo/".

7 years agoExtract out public methods of Repository into an abstract base class
Raja R Harinath [Thu, 22 Jul 2010 12:07:07 +0000 (17:37 +0530)]
Extract out public methods of Repository into an abstract base class

Rename Repository to FastImportRepository.  We will be introducing new types soon.

7 years agoclear out a local that is re-used
Raja R Harinath [Thu, 22 Jul 2010 12:06:33 +0000 (17:36 +0530)]
clear out a local that is re-used

7 years agoMove backup branches and tags to refs/branches/
Raja R Harinath [Tue, 20 Jul 2010 19:31:48 +0000 (01:01 +0530)]
Move backup branches and tags to refs/branches/

The naming scheme is

  refs/backups/r<svn-revision>/(heads|tags)/<branch>

Where <svn-revision> is the revision where the branch is being reset:
either for deletion or to be overwritten.

We use a separate namespace so that we don't clutter up branch-name lists
and tag lists with deleted tags.  These refs will keep the commits alive
as far as 'git gc' is concerned, but will be fairly unobstrusive otherwise.

7 years agoCarve out Repository::markFrom from createBranch(), and noteCopyFromBranch()
Raja R Harinath [Tue, 20 Jul 2010 19:13:22 +0000 (00:43 +0530)]
Carve out Repository::markFrom from createBranch(), and noteCopyFromBranch()

Put the slightly tricky qUpperBound logic in only one place.

7 years agoMake Repository::createBranch use resetBranch now that they're similar
Raja R Harinath [Tue, 20 Jul 2010 17:43:16 +0000 (23:13 +0530)]
Make Repository::createBranch use resetBranch now that they're similar

7 years agoRe-arrange Repository::createBranch to be closer to resetBranch
Raja R Harinath [Tue, 20 Jul 2010 16:38:24 +0000 (22:08 +0530)]
Re-arrange Repository::createBranch to be closer to resetBranch

While at it, make progress message for the "unknown from" case similar
to the normal progress message.  This ensures that any reconstructed
branch data-structure will not lose any information.

7 years agoAdd a resetBranch() to help refactor createBranch and deleteBranch.
Raja R Harinath [Tue, 20 Jul 2010 15:47:36 +0000 (21:17 +0530)]
Add a resetBranch() to help refactor createBranch and deleteBranch.

Only deleteBranch uses it for now.

7 years agoRemove exit(1) in createBranch. Return EXIT_SUCCESS/EXIT_FAILURE instead
Raja R Harinath [Tue, 20 Jul 2010 15:34:29 +0000 (21:04 +0530)]
Remove exit(1) in createBranch.  Return EXIT_SUCCESS/EXIT_FAILURE instead

Allow graceful exit of all fast-import processes when createBranch fails.
For consistency, add return value to deleteBranch, even though it always
returns EXIT_SUCCESS.

7 years agoMake the new incremental mode permanent
Raja R Harinath [Mon, 19 Jul 2010 13:44:03 +0000 (19:14 +0530)]
Make the new incremental mode permanent

With the changes made to Repository::reloadBranches when --incremental
was introduced, the older mode wasn't working, anyway.

7 years agoHandle branching better for branches that use "prefix" rules
Raja R Harinath [Sun, 11 Jul 2010 19:16:33 +0000 (00:46 +0530)]
Handle branching better for branches that use "prefix" rules

When creating branches/tags off a branch that use "prefix" rules,
svn2git used to create new commits rather than just copying the git
ref over.  This was due to SvnRevision::exportInteral() using
'path.isEmpty()' to determine if a change related to the whole branch
or not.  If a "prefix" rule is used, then 'path' will not be empty.

We change it to instead look at how much of the change string was
chomped up by the rule match.  If the whole string was chomped up, it
means that the change describes the whole branch.

7 years agoMake error handling of --incremental and --resume-from idempotent
Raja R Harinath [Sun, 11 Jul 2010 13:38:53 +0000 (19:08 +0530)]
Make error handling of --incremental and --resume-from idempotent

When --resume-from failed in incremental mode, the log files that detected
the error condition were truncated.  So, if the same command line was executed
again, the invocation would go through.

We now restore the log files from backup when we detect we're going to fail.
The restored log files may not all be the same as we originally started with,
but we only truncate information that would anyway be truncated on the
next successful run.

7 years agomake --incremental robust to inconsistent import directories
Raja R Harinath [Sat, 10 Jul 2010 13:53:07 +0000 (19:23 +0530)]
make --incremental robust to inconsistent import directories

An interrupted import (say with Ctrl-C) can leave the import directory in an
inconsistent state.  This can be due to checkpointing fast-import only
occassionally, but updating log-* files immediately, and/or other reasons.

The incremental mode can detect certain such situations and rewind back to a
safe state.  Note that since the default commit-interval is quite large, this
rewind can end up backtracking a lot.

Note also that import interrupted under the control of svn2git, say, for
missing rules should leave the import directory in a consistent state for
the purpose of svn2git.

7 years agoReduce size of fast-import marks file by not persisting file-level marks
Raja R Harinath [Sat, 10 Jul 2010 10:50:04 +0000 (16:20 +0530)]
Reduce size of fast-import marks file by not persisting file-level marks

Use two allocators for marks, one persistent commit counter that starts at 0
and counts up commits, and a transitional counter, for files, that counts down
from maxMark, and is reset on each SVN revision.

Note that the marks file will still have marks for some, but not all, files.
The number of such marks is limited by the size of the SVN revision that affects
the most files.

For instance, this changed the size of one marks file from 19M to 3.2M.

fast-import issues: We currently set maxMark = (1<<20)-1.  Anything large seems
to trigger a bug in the sparse array dumping routine in git-fast-import in
certain versions of git.

7 years agoEnsure deleteBranch() actually writes to git-fast-import
Raja R Harinath [Fri, 9 Jul 2010 16:40:57 +0000 (22:10 +0530)]
Ensure deleteBranch() actually writes to git-fast-import

Use startFastImport to start off the git-fast-import process if necessary.

7 years agoIntroduce incremental mode with --incremental flag
Raja R Harinath [Fri, 9 Jul 2010 14:31:36 +0000 (20:01 +0530)]
Introduce incremental mode with --incremental flag

We use the progress logs that we carefully maintained to recreate the
branch data structures, as described in earlier commits.

A major change/improvement is that reloadBranches() now uses the marks file
and internal data structures to prime the fast-import rather than using
git-rev-parse.

We also handle --resume-from properly, by truncating the log file to revisions
that only precede the revision resumed from.  Note that git fast-import allows
marks to be reused without any extra processing.

7 years agoHandle SVN directory deletes that lead to branch deletes
Raja R Harinath [Tue, 6 Jul 2010 17:38:23 +0000 (23:08 +0530)]
Handle SVN directory deletes that lead to branch deletes

SVN directory deletes often indicate one or more branch deletions.  However,
since the deleted directory isn't present in the resulting revision, several
of the indicators used by the rule-application mechanism aren't present.
This forces us to introduce several useless dummy rules to avoid errors.

We now note deletions and use the previous revision to determine several
properties, including whether the deleted item is a directory or not, and to
enumerate the contents of the directory in recurse mode.

We add an additional heuristic for unknown repositories -- i.e., when a rule
fired, but it's repository was invalid.  We recurse in this case hoping to
catch a more specific rule.  I believe this is safe: because some other rule
must've seen the same directory before, when it or a subdirectory was created,
and decided _not_ to create a repository at that point -- so recursing and/or
ignoring the contents of the just deleted directory won't corrupt the history,
it can only improve it.

We use mark :0 to note mark deletions internally, and in the progress logs.

(Note that cvs2svn creates wierd commits where a whole tree is copied first,
and then subtrees are pruned.  In such cases, neither the previous revision
nor the current revision have the deleted directory -- we ignore this case
as before.  There's no information loss since the final contents of the revision
are exactly what is desired.)

7 years agoMove SVN revision to fast-import mark mapping to the per-branch datastructure.
Raja R Harinath [Mon, 5 Jul 2010 16:06:28 +0000 (21:36 +0530)]
Move SVN revision to fast-import mark mapping to the per-branch datastructure.

A single SVN revision can affect multiple branches in the same repository.
Keeping track of only one mark per revision loses information and makes
the history incorrect.

7 years agoProperly implement the 16-parent limit, rather than conservatively stopping at
Raja R Harinath [Mon, 5 Jul 2010 16:05:10 +0000 (21:35 +0530)]
Properly implement the 16-parent limit, rather than conservatively stopping at
15 commit parents.

7 years agoDon't skip same branch revisions when inferring multiple merge parents.
Raja R Harinath [Thu, 1 Jul 2010 18:05:38 +0000 (23:35 +0530)]
Don't skip same branch revisions when inferring multiple merge parents.

Experience with the mono tree shows that it isn't too annoying, and
there might be some useful history hidden in there.

7 years agoInfer some copy sources as additional parents
Raja R Harinath [Thu, 1 Jul 2010 12:56:26 +0000 (18:26 +0530)]
Infer some copy sources as additional parents

We use a literal meaning of multiple commit parents to allow us to infer
some partial repository copying as merges.  This helps us

  1) track history despite some directory reorganization
  2) link subset commits to parents
  3) infer some merges which were achieved by overwriting
     a subtree with contents from another branch

This seems to work well enough even with cvs2svn monster commits.  The
heuristics have been tuned by gut feel to work reasonably well with mono's
SVN repository. They can definitely be improved.

7 years agoExit with EXIT_FAILURE if svn.exportRevision() failed
Raja R Harinath [Thu, 1 Jul 2010 02:55:49 +0000 (08:25 +0530)]
Exit with EXIT_FAILURE if svn.exportRevision() failed

7 years agoHandle same branch copies using createBranch to allow reseating
Raja R Harinath [Thu, 1 Jul 2010 02:54:52 +0000 (08:24 +0530)]
Handle same branch copies using createBranch to allow reseating
branch tip to an older revision.

7 years agoMerge branch 'master' of git://gitorious.org/svn2git/svn2git
Torgny Nyblom [Sun, 27 Jun 2010 17:13:17 +0000 (19:13 +0200)]
Merge branch 'master' of git://gitorious.org/svn2git/svn2git

8 years agoAdd --force to 'git fast-import' command-line
Raja R Harinath [Mon, 21 Jun 2010 13:38:23 +0000 (19:08 +0530)]
Add --force to 'git fast-import' command-line

Now that we support arbitrarily reseating branches, the branch can be
reset to older SVN revisions or derive from a different branch point.

The "only fast-forward" rule of git branch imports gets in our way.
Note that the application of the rule is fairly unpredictable since the
checking happens only at 'checkpoint's.

Let's trust our handling of SVN history and override the sanity-check.
Note that we don't lose any information since reseated branches are
snapshotted before reset.

8 years agoAllow comments (start with '#') in accounts map
José Manuel Santamaría Lema [Mon, 21 Jun 2010 14:07:47 +0000 (16:07 +0200)]
Allow comments (start with '#') in accounts map

8 years agoFix type error
Raja R Harinath [Sun, 20 Jun 2010 18:58:51 +0000 (00:28 +0530)]
Fix type error

8 years agoImprove progess log
Raja R Harinath [Sun, 20 Jun 2010 17:29:41 +0000 (22:59 +0530)]
Improve progess log

Make the progress log for commit and branch creation more uniform and
machine parse-able.  Add 'mark' information to log so that it can be
cross-referenced against the fast-import marks file.

The log-* files now have enough information in a convenient enough form
that the export can eventually be made fully resumable and incremental.
The format is essentially

  progress SVN r<svn-rev> branch <git-branch> = (:<mark> | <other-git-branch>)

with a possible trailing # comment.  Any line not matching this can be

The incremental mode would prime the internal structures with

  if (<mark>) repository->commitMarks[<svn-rev>] = <mark>;
  repository->branches[<git-branch>].commits.append(<svn-rev>);

8 years agoTrack commits per-branch
Raja R Harinath [Sun, 20 Jun 2010 13:45:39 +0000 (19:15 +0530)]
Track commits per-branch

Previously, all the SVN commits were tracked in a linear array, and we searched
for nearest commits in that array.  However, SVN history is not linear, and the
'next smallest commit' search was picking the wrong commit.

I've moved the commit array into the 'Branch' structure.

As a minor subtlety, the branch creation revision is also noted in the
'commitMarks' structure, by copying the commit mark of the branch point.
We need this to ensure that the commits array is strictly non-decreasing.