What do you mean by

USER wrote:

opening a file from a library

If you want to load a custom library in OMEdit you'd typically use the File/Load Library command from the menu bar. This will allow you to brwose to the destination of your custom library to open the contained package.mo file or if the library is stored in one file, then open the <lib_name>.mo file. This way the working directory is not changed by the library will show up in the class browser in OMEdit.

All libraries stored under the MODELICAPATH are found automatically.

In Dymola one can easily add its own search paths. Is there a similar way for OMC to extend the search path?

Env.variable OPENMODELICALIBRARY can be used