Restoring OneNote data from the local cache (08 Oct 2017)

This is how it all started: After withstanding the constant nagging for a while, I finally gave in to those prompts to upgrade to Office 2016 on my developer laptop. During the upgrade, OneNote 2016 warned me that it may lose those few notes which had not been synced to the cloud yet. This is because OneNote 2016 will not use an older installation's local cache, but instead create a new empty local cache during installation, which will then fill incrementally by downloading notes from the cloud.

I shrugged off the warning because I was quite certain I had not made any significant changes recently, so everything should have been synced already. Of course I was totally wrong.

But after installation, OneNote 2016 would not display about two months worth of notes even though I had taken them (in OneNote 2013) on the very system on which I performed the upgrade. It turned out that OneNote 2013 had indeed not synced any of those notes to the cloud - for a period of almost two months.

The bad news is that I will probably never find out why and how synchronisation failed for such a long time, and why I never noticed any warnings about it. I guess I will be quite paranoid about synchronisation for a while now.

But the good news is that I managed to restore my notes. OneNote 2016 had created a new and empty local cache below %LOCALAPPDATA%\Microsoft\OneNote\16.0, but it had not deleted the old OneNote 2013 cache (below %LOCALAPPDATA%\Microsoft\OneNote\15.0), and this saved my bacon.

Others have fallen into the same or similar traps before, of course, and so there are a number of related discussions out there on the topic, for example at https://answers.microsoft.com/en-us/msoffice/forum/msoffice_onenote-mso_other-mso_2010/recover-information-from/8bf30713-316b-49cb-abc3-a8ce8e4b310d. A number of approaches are mentioned, such as:

• Restore OneNote sections from the C:\Users\<name>\AppData\Local\Microsoft\OneNote\15.0\Backup directory
  • This worked only partially because the latest backup was already several days old.
• Extract notes from the old OneNoteOfflineCache.onecache file in C:\Users\<name>\AppData\Local\Microsoft\OneNote\15.0\ by running onenote.exe /forcerepair on it.
  • This would fail because both OneNote 2013 and OneNote 2016 would complain that the cache file was of a format written by a "future version" of OneNote. (Which wasn't true, because the local cache file was written by OneNote 2013.
  • See also https://support.microsoft.com/en-us/help/825779/how-to-recover-information-in-a-damaged-onenote-section-file

The approach which did work in the end was as follows:

• I installed OneNote 2013 on a separate Windows VM.
• Then I copied over the cache files from my developer laptop to the Windows VM, i.e. both the OneNoteOfflineCache.onecache file and the OneNoteOfflineCache_Files directory (which holds all the attachments), overwriting the default local cache files of the OneNote 2013 installation on the VM.
• After starting OneNote 2013 on the VM, it displayed all notes just fine. Big sigh of relief.
• Syncing those notes from the VM to the cloud would not work, though. I first had to move all the notes to a new section in the affected notebook, and then wait until all notes had been synced.
• And now, finally, the notes reappeared in my OneNote 2016 installation on my developer laptop as well.

I also could have uninstalled OneNote 2016 on my developer laptop and replaced it with the older OneNote 2013, and in fact I tried, but the OneNote 2013 installer told me to uninstall all of Office 2016 first, from which I shied away.

macOS Sierra auf meinem Macbook Pro (21 Oct 2016)

Denn gleich nach der Installation stellte ich fest, dass mein USB-Ethernet-Adapter entweder nicht mehr erkannt wurde oder aber zumindest keine Netzwerkverbindung mehr zustande brachte. In meinem Adapter steckt der Ethernet-Controller AX88179. Unter diesem Stichwort fanden sich schnell Diskussionen wie die unter https://forums.developer.apple.com/thread/48775, und so war schnell klar, dass eine Neuinstallation des zugehörigen Treibers fällig war. Und da ich so lange gewartet hatte, war zum Glück auch schon eine Treiberversion vorrätig, die mein Problem schnell löste:

Ausserdem musste ich feststellen, daß Paintbrush nicht mehr starten will. Offenbar gibt es dazu auch keine neuere Version, und so bin ich jetzt auf der Suche nach gleichwertigem oder besseren Ersatz.

Nachtrag (1. Juni 2017): Nachdem ich heute Paintbrush testweise neu heruntergeladen habe, lief es scheinbar anstandslos, und das trotz macOS Sierra. Mindestens in diesem Punkt also Entwarnung.

CoCreate Modeling: Wie startet man ein interaktives cmd.exe? (05 Jul 2016)

Es liegt nahe, das zunächst mit

  (oli:sd-sys-exec "cmd.exe")

zu versuchen. Das führt dann aber dazu, dass CoCreate Modeling scheinbar hängt und nichts Erkennbares passiert.

cmd.exe ist ein Kommandozeilenprogramm. Deswegen ist es völlig normal, dass ("grafisch") nichts passiert, wenn man cmd.exe als externes Programm ohne Parameter startet, zum Beispiel per sd-sys-exec. Dann wartet cmd.exe nämlich einfach im Hintergrund auf weitere Eingaben und tut sonst nichts.

Will man cmd.exe in einem eigenen Terminalfenster (landläufig "DOS-Fenster" oder "command shell" oder "command prompt") starten und interaktiv laufen lassen, kann man das so erreichen:

  (oli:sd-sys-exec "start cmd.exe")

(Zu den Kommandozeilenparametern und Besonderheiten des Helferleins start siehe http://ss64.com/nt/start.html.)

Bonusfrage: Wenn cmd.exe ein Kommandozeilenprogramm ohne grafische Oberfläche ist, wieso öffnet sich denn ein Terminalfenster, wenn man cmd.exe aus Windows Explorer heraus startet?

Antwort: Weil Explorer entsprechend vorkonfiguriert ist - intern wird in so einem Fall nicht einfach nur cmd.exe ausgeführt, sondern das moralische Äquivalent zu start cmd.exe.

Bonusfrage 2: Woher weiss Windows eigentlich, wo cmd.exe liegt? Muss man da nicht einen Pfad wie C:\Windows\System32\cmd.exe angeben?

Hintergrund: In der Forumsfrage wurde ein solcher hartkodierter Pfad verwendet.

Antwort: Das Verzeichnis, in dem cmd.exe liegt, taucht im Inhalt der Umgebungsvariablen PATH auf, die Windows beim Starten von Programmen konsultiert. Damit ist die explizite Angabe eines Pfades unnötig. Mehr noch, sie ist sogar kontraproduktiv und fehlerträchtig - denn nicht auf jedem Rechner liegt das Windows-Verzeichnis unter C:\Windows.

Bonusfrage 3: Wozu ist das eigentlich gut, so eine interaktive Instanz von cmd.exe aus einer CAD-Applikation heraus zu starten?

Kopfkratzende erste Antwort: Für sachdienliche Hinweise dankbar

Zweite Antwort nach Eintreffen der angeforderten sachdienlichen Hinweise: Ziel war es offenbar letztlich, ein kleines interaktives Kommandozeilenprogramm zu starten - der Start von cmd.exe war nur erster Test dafür.

Projekt Umstieg (02 Jul 2016)

Die Aussicht, die wunderschöne Parkfläche im Mittleren Schlossgarten zu restaurieren; dabei einen leistungsfähigen Kopfbahnhof zurückzubekommen, der verschiedenste Verkehrsarten integriert (Fahrrad, Bus, Auto, Bahn, S-Bahn); damit eine riesige Menge erneuerbare Energie zu erzeugen; und schliesslich auch noch viele Jahre Bauzeit und Milliarden an Kosten zu sparen: Was könnte man daran nicht gut finden?

A poor man's Common Lisp profiler (08 Mar 2016)

Alex's original code used quite some Lisp magic, including the little-known symbol-function which I elaborated about long time ago. But the code did not quite work yet. I gladly took the challenge. and ended up with a few lines of Lisp code which could profile (almost) any Lisp function in the system. The technique I used was to wrap the original function definition in a lambda closure. That closure is then installed using symbol-function.

(in-package :clausbrod.de)
(export '(profile-function unprofile-function list-profiling-results))

(let ((profile-hashtable (make-hash-table)))
  (defun profile-function(func)
    "Instrument function for profiling"

    ;; check if symbol-plist already contains profiler flag
    (unless (get func :profile-original-symbol-function)
      (let ((original-symbol-function (symbol-function func)))
   (when original-symbol-function
     (setf (get func :profile-original-symbol-function) original-symbol-function) ;; mark as profiled

     ;; install profiler code
     (setf (symbol-function func)
      (lambda(&rest r)
        (let ((start-time (f2::seconds-since-1970)))
          (unwind-protect
         (if r
             (apply original-symbol-function r)
           (funcall original-symbol-function))
            (let ((execution-time (- (f2::seconds-since-1970) start-time))
             (accum (gethash func profile-hashtable)))
         (if accum
             (setf (gethash func profile-hashtable) (+ accum execution-time))
           (setf (gethash func profile-hashtable) execution-time))
         (format *standard-output* "~%Execution time for ~S: ~,10F~%" func execution-time))))))
     ))))

  (defun unprofile-function(func)
    "Remove profiling instrumentation for function"
    (let ((original-symbol-function (get func :profile-original-symbol-function)))
      (when (remprop func :profile-original-symbol-function)
   (setf (symbol-function func) original-symbol-function))))

  (defun list-profiling-results()
    "List profiling results in order of decreasing accumulated execution times"
    (format *standard-output* "~%Accumulated execution times:~%")
    (let (table-as-list)
      (maphash (lambda(k v) (push (cons k v) table-as-list)) profile-hashtable)
      (dolist (pair (sort table-as-list #'> :key #'cdr))
   (format *standard-output* "~S:  ~,10F~%" (car pair) (cdr pair)))))
  )

(f2::win-open-console-window)
(setf si::*enter-break-handler* t)
(use-fast-links nil)

There are other profilers out there for Common Lisp, but it is not always straightforward to make them work in CoCreate Modeling which implements a subset of CLtL1 only. So who knows, maybe someone out there will actually find this useful!

To profile a function:

  (clausbrod.de:profile-function 'my-function)

Now execute my-function at your heart's content. Every time the function is called, the profiler measures its execution time. When the test session is completed, accumulated execution times can be listed as follows:

  (clausbrod.de:list-profiling-results)

And here is how to profile all functions in a given Lisp package:

  (do-external-symbols (s (find-package "FOO"))
    (when (function s)
      (clausbrod.de:profile-function s)))

My implementation differs almost entirely from Alex' version, which allows me to call it my own, but of course I owe thanks to Alex for starting the discussion in the forum and posting his original inspirational code!

The code is now available as a Github project, see https://github.com/clausb/lisp-profiler. There is even a simple GUI dialog on top of the low-level profiling code:

The version of the code shown above uses a SolidDesigner-specific way of getting the current time in high precision. The improved version in the Github project should work in other Lisp dialects as well. Fingers crossed.

A poor man's Common Lisp profiler (08 Mar 2016)

Alex's original code used quite some Lisp magic, including the little-known symbol-function which I elaborated about long time ago. But the code did not quite work yet. I gladly took the challenge. and ended up with a few lines of Lisp code which could profile (almost) any Lisp function in the system. The technique I used was to wrap the original function definition in a lambda closure. That closure is then installed using symbol-function.

(in-package :clausbrod.de)
(export '(profile-function unprofile-function list-profiling-results))

(let ((profile-hashtable (make-hash-table)))
  (defun profile-function(func)
    "Instrument function for profiling"

    ;; check if symbol-plist already contains profiler flag
    (unless (get func :profile-original-symbol-function)
      (let ((original-symbol-function (symbol-function func)))
   (when original-symbol-function
     (setf (get func :profile-original-symbol-function) original-symbol-function) ;; mark as profiled

     ;; install profiler code
     (setf (symbol-function func)
      (lambda(&rest r)
        (let ((start-time (f2::seconds-since-1970)))
          (unwind-protect
         (if r
             (apply original-symbol-function r)
           (funcall original-symbol-function))
            (let ((execution-time (- (f2::seconds-since-1970) start-time))
             (accum (gethash func profile-hashtable)))
         (if accum
             (setf (gethash func profile-hashtable) (+ accum execution-time))
           (setf (gethash func profile-hashtable) execution-time))
         (format *standard-output* "~%Execution time for ~S: ~,10F~%" func execution-time))))))
     ))))

  (defun unprofile-function(func)
    "Remove profiling instrumentation for function"
    (let ((original-symbol-function (get func :profile-original-symbol-function)))
      (when (remprop func :profile-original-symbol-function)
   (setf (symbol-function func) original-symbol-function))))

  (defun list-profiling-results()
    "List profiling results in order of decreasing accumulated execution times"
    (format *standard-output* "~%Accumulated execution times:~%")
    (let (table-as-list)
      (maphash (lambda(k v) (push (cons k v) table-as-list)) profile-hashtable)
      (dolist (pair (sort table-as-list #'> :key #'cdr))
   (format *standard-output* "~S:  ~,10F~%" (car pair) (cdr pair)))))
  )

(f2::win-open-console-window)
(setf si::*enter-break-handler* t)
(use-fast-links nil)

There are other profilers out there for Common Lisp, but it is not always straightforward to make them work in CoCreate Modeling which implements a subset of CLtL1 only. So who knows, maybe someone out there will actually find this useful!

To profile a function:

  (clausbrod.de:profile-function 'my-function)

Now execute my-function at your heart's content. Every time the function is called, the profiler measures its execution time. When the test session is completed, accumulated execution times can be listed as follows:

  (clausbrod.de:list-profiling-results)

And here is how to profile all functions in a given Lisp package:

  (do-external-symbols (s (find-package "FOO"))
    (when (function s)
      (clausbrod.de:profile-function s)))

My implementation differs almost entirely from Alex' version, which allows me to call it my own, but of course I owe thanks to Alex for starting the discussion in the forum and posting his original inspirational code!

The code is now available as a Github project, see https://github.com/clausb/lisp-profiler. There is even a simple GUI dialog on top of the low-level profiling code:

The version of the code shown above uses a SolidDesigner-specific way of getting the current time in high precision. The improved version in the Github project should work in other Lisp dialects as well. Fingers crossed.

CoCreate Modeling: Changing the current directory during startup (02 Nov 2015)

In the article, I showed how to subscribe to events in CoCreate Modeling (aka "PTC Creo Elements/Direct Modeling") for fun and profit. It turns out that this technique can also be applied to solve the following problem: In customization code which is loaded into CoCreate Modeling during startup, you want to set the user's default working directory to some default value specific to your environment or team.

You'd think that this should be trivial, as the current directory can be set using the IKIT's sd-set-current-working-directory API. But when you call this function during startup (i.e. from code in sd_customize, or in code loaded from there), you may find that other customization code or even CoCreate Modeling itself changes the current directory after your code runs. This is because CoCreate Modeling remembers the directory which was current before the user closed the last session. When you restart the application, it will try to "wake up" in precisely that working directory.

To override this behavior, here's a simple trick:

• In sd_customize (or, preferably, in code loaded from there), register an event handler for the SD-INTERACTIVE-EVENT.
• This event will be fired when startup has completed and the application becomes interactive.
• In the event handler:
  • Set the current working directory as you see fit
  • Unregister from the event (we want one-shot behavior here)

And here is what event handler code like this would look like:

(in-package :de.clausbrod)
(use-package :oli)

(defun interactive-event-handler(&rest r)
  (sd-set-current-working-directory (user-homedir-pathname))
  (sd-unsubscribe-event *SD-INTERACTIVE-EVENT* 'interactive-event-handler))

(sd-subscribe-event *SD-INTERACTIVE-EVENT* 'interactive-event-handler)

This particular event handler sets the current working directory to the user's home directory, but this is of course just an example for a reasonable default.

Older topics...

• 2015

Add a new blog entry.

to top