AutoPkg: Crowd-sourcing Mac packaging and deployment

Thanks to everyone who attended Greg’s and my talk today at MacSysAdmin 2014 in Göteborg! Again, I give my sincere thanks to Tycho and all those who organize the fantastic MacSysAdmin conference every year. I’m honoured to be among such great company, speakers and attendees.

Here are links to the resources that were linked to in the slides:

AutoPkg:

Getting support:

Other tools:

Work by Shea Craig::

AutoPkg integration for other software management systems:

Previous video sessions by Greg:

Articles on AutoPkg in MacTech Magazine, print and iPad:

  • Issues October, November 2013 (intros to AutoPkg)
  • Issues 2014.342, 2014.343 (writing custom Processors)

Using Charles Web Proxy to figure out the built-in updaters in Mac apps:

Leave a comment

Building native extensions since LLVM 5.1

With LLVM / clang 5.1, Apple introduced a change where any unrecognized command option causes a hard failure. Unfortunately, there are many packages in the Python package index that have not yet adapted to this change when building on OS X and include unsupported flags (in my experience it’s usually been -mno-fused-madd). I first started running into this frequently when installing some Python packages using pip. This can also be an issue for other package managers like RubyGems.

Output like this is common:

➜ pip install lxml

building 'lxml.etree' extension
creating build/temp.macosx-10.9-intel-2.7
creating build/temp.macosx-10.9-intel-2.7/src
creating build/temp.macosx-10.9-intel-2.7/src/lxml
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/Users/tsutton/venv/lxml-test/build/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace

clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]
clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
error: command 'cc' failed with exit status 1

Sometimes packages will still install and fall back to slower, non-native packages. Some packages, like lxml, simply won’t install.

Luckily there is a straightforward workaround: define the CFLAGS environment variable and pass it the compatibility option -Wunused-command-line-argument-hard-error-in-future when we execute the command. The C compilers look for this and include these additional arguments. Like this:

➜ CFLAGS="-Wunused-command-line-argument-hard-error-in-future" pip install lxml

...

cc -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -Wunused-command-line-argument-hard-error-in-future -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/Users/tsutton/venv/lxml-test/build/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace
cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. -Wunused-command-line-argument-hard-error-in-future build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/etree.so
building 'lxml.objectify' extension
cc -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -Wunused-command-line-argument-hard-error-in-future -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/Users/tsutton/venv/lxml-test/build/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -w -flat_namespace
cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. -Wunused-command-line-argument-hard-error-in-future build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/objectify.so

Successfully installed lxml
Tagged , , , | Leave a comment