Mcuboot hack to prevent `newt upgrade` failure
For projects using mcuboot, `newt upgrade` has been failing. The
error message is:
Error: Error updating "mcuboot": error: The following untracked working
tree files would be overwritten by checkout:
ext/mbedtls/include/mbedtls/asn1.h
ext/mbedtls/include/mbedtls/bignum.h
ext/mbedtls/include/mbedtls/check_config.h
ext/mbedtls/include/mbedtls/config.h
ext/mbedtls/include/mbedtls/ecdsa.h
ext/mbedtls/include/mbedtls/ecp.h
ext/mbedtls/include/mbedtls/md.h
ext/mbedtls/include/mbedtls/oid.h
ext/mbedtls/include/mbedtls/pk.h
ext/mbedtls/include/mbedtls/platform.h
ext/mbedtls/include/mbedtls/platform_util.h
ext/mbedtls/include/mbedtls/threading.h
Please move or remove them before you switch branches.
Aborting
The problem is caused by a recent change to mcuboot - a normal directory
was replaced with a submodule. Git is unable to transition from a
post-replace commit to a pre-replace commit because some files in the
submodule used to exist in the mcuboot repo itself.
This commit implements a pair of hacks to workaround this problem. Upon
performing a checkout operation on the mcuboot repo:
1. If newt detects the above error message, it removes the `ext/mbedtls`
directory and tries again.
2. If newt detects a different warning message (`unable to rmdir
'sim/mcuboot-sys/mbedtls': Directory not empty`), it removes
`sim/mcuboot-sys/mbedtls`.
The second hack is required when transitioning from pre-replace to
post-replace. In this case, the checkout operation succeeds, but an
orphaned directory is left behind, making the repo look dirty.
This is expected to be a short-term fix. When we have a general fix
for this issue, this hack can be removed.
diff --git a/newt/downloader/downloader.go b/newt/downloader/downloader.go
index a5af7bf..8b3a607 100644
--- a/newt/downloader/downloader.go
+++ b/newt/downloader/downloader.go
@@ -365,6 +365,87 @@
return gd.init(path)
}
+// untrackedFilesFromCheckoutErr collects the list of untracked files that
+// prevented a checkout from succeeding. It returns nil if the provided error
+// does not indicate that untracked files are in the way.
+func untrackedFilesFromCheckoutErr(err error) []string {
+ var files []string
+
+ text := err.Error()
+ lines := strings.Split(text, "\n")
+
+ collecting := false
+ for _, line := range lines {
+ if !collecting {
+ if strings.Contains(line,
+ "The following untracked working tree files would "+
+ "be overwritten by checkout:") {
+ collecting = true
+ }
+ } else {
+ if strings.Contains(line, "Please move or remove them before") {
+ collecting = false
+ } else {
+ files = append(files, strings.TrimSpace(line))
+ }
+ }
+ }
+
+ return files
+}
+
+// applyMcubootPreHack attempts to clean up the mcuboot repo so that the
+// subsequent checkout operation will succeed. This hack is required because a
+// directory in the mcuboot repo was replaced with a submodule. Git is unable
+// to transition from a post-replace commit to a pre-replace commit because
+// some files in the submodule used to exist in the mcuboot repo itself. If
+// this issue is detected, this function deletes the submodule directory so
+// that the checkout operation can be attempted again.
+func applyMcubootPreHack(repoDir string, err error) error {
+ if !strings.HasSuffix(repoDir, "repos/mcuboot") {
+ // Not the mcuboot repo.
+ return err
+ }
+
+ // Check for an "untracked files" error.
+ files := untrackedFilesFromCheckoutErr(err)
+ if len(files) == 0 {
+ // Not a hackable error.
+ return err
+ }
+
+ for _, file := range files {
+ if !strings.HasPrefix(file, "ext/mbedtls") {
+ return err
+ }
+ }
+
+ path := repoDir + "/ext/mbedtls"
+ log.Debugf("applying mcuboot hack: removing %s", path)
+ os.RemoveAll(path)
+ return nil
+}
+
+// applyMcubootPostHack attempts to clean up the mcuboot repo so that the
+// subsequent checkout operation will succeed. This hack is required because a
+// directory in the mcuboot repo was replaced with a submodule. This hack
+// should be applied after a successful checkout from a pre-replace commit to a
+// post-replace commit. This function deletes an orphan directory left behind
+// by the checkout operation.
+func applyMcubootPostHack(repoDir string, output string) {
+ if !strings.HasSuffix(repoDir, "repos/mcuboot") {
+ // Not the mcuboot repo.
+ return
+ }
+
+ // Check for a "unable to rmdir" warning (pre- to post- submodule move).
+ if strings.Contains(output, "unable to rmdir 'sim/mcuboot-sys/mbedtls'") {
+ path := repoDir + "/sim/mcuboot-sys/mbedtls"
+ log.Debugf("applying mcuboot hack: removing %s", path)
+ os.RemoveAll(path)
+ }
+}
+
func (gd *GenericDownloader) Checkout(repoDir string, commit string) error {
// Get the hash corresponding to the commit in case the caller specified a
// branch or tag. We always want to check out a hash and end up in a
@@ -380,8 +461,17 @@
hash,
}
- if _, err := executeGitCommand(repoDir, cmd, true); err != nil {
- return err
+ o, err := executeGitCommand(repoDir, cmd, true)
+ if err != nil {
+ if err := applyMcubootPreHack(repoDir, err); err != nil {
+ return err
+ }
+
+ if _, err := executeGitCommand(repoDir, cmd, true); err != nil {
+ return err
+ }
+ } else {
+ applyMcubootPostHack(repoDir, string(o))
}
// Always initialize and update submodules on checkout. This prevents the
