Building native extensions since LLVM 5.1

With LLVM / clang 5.1, Apple introduced a change where any unrecognized command option causes a hard failure. Unfortunately, there are many packages in the Python package index that have not yet adapted to this change when building on OS X and include unsupported flags (in my experience it’s usually been -mno-fused-madd). I first started running into this frequently when installing some Python packages using pip. This can also be an issue for other package managers like RubyGems.

Output like this is common:

➜ pip install lxml

building 'lxml.etree' extension
creating build/temp.macosx-10.9-intel-2.7
creating build/temp.macosx-10.9-intel-2.7/src
creating build/temp.macosx-10.9-intel-2.7/src/lxml
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/Users/tsutton/venv/lxml-test/build/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace

clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]
clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
error: command 'cc' failed with exit status 1

Sometimes packages will still install and fall back to slower, non-native packages. Some packages, like lxml, simply won’t install.

Luckily there is a straightforward workaround: define the CFLAGS environment variable and pass it the compatibility option -Wunused-command-line-argument-hard-error-in-future when we execute the command. The C compilers look for this and include these additional arguments. Like this:

➜ CFLAGS="-Wunused-command-line-argument-hard-error-in-future" pip install lxml


cc -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -Wunused-command-line-argument-hard-error-in-future -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/Users/tsutton/venv/lxml-test/build/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace
cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. -Wunused-command-line-argument-hard-error-in-future build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/
building 'lxml.objectify' extension
cc -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -Wunused-command-line-argument-hard-error-in-future -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/Users/tsutton/venv/lxml-test/build/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -w -flat_namespace
cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. -Wunused-command-line-argument-hard-error-in-future build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/

Successfully installed lxml
Tagged , , , | Leave a comment

How to Package Profiles

Pkg_256.pngPart of a managed Mac’s configuration is often one or more Profiles, either Configuration Profiles, or an Enrollment Profile for an MDM server like Apple’s Profile Manager or Cisco Meraki Systems Manager.

There are multiple ways to install these. You can have users double-click and install these .mobileconfig files themselves via a website or e-mail if they have administrative rights on their machines. You can have DeployStudio install them as part of a workflow and not care how it’s done, or have a management service like the Casper Suite configure and manage them for clients (and again, not need to care how it’s done).

But if we want the most portable way possible to install a profile on a Mac, it might be the simplest to fall back to the Mac’s lingua franca of software configuration (for better or for worse): the Installer Package. A profile that can be installed via a package install can be installed with any management software (even Apple Remote Desktop, version 3 turning 8 years old tomorrow). And like a profile, it can simply be opened and installed like any other piece of software by a technician or user.

This question comes up often enough for people using Munki, or other management systems that don’t have some kind of purpose-built mechanism for dealing with profiles. Generally these are the steps:

  1. Have the package install the profile somewhere like any other file.
  2. Run the command /usr/bin/profiles (as root) in a postinstall script to install this profile.
  3. There’s no step three.

Actually, there is if you’d like to also include a mechanism to remove the profile: You’d want to write some short script that would remove the profile, as well as configure the system so that it can know that this profile is no longer installed. Since we’re using an installer package, we have the benefit of being able to check for a receipt of the package.

You also might like to install this profile on an non-booted volume (either a clean image built with something like AutoDMG, or a Mac system connected via Target Disk Mode). But since we don’t have profiles available, we actually want to install it to a special place that OS X looks for at boot time for any profiles to install: /private/var/db/ConfigurationProfiles/Setup, as well as clear a special .profileSetupDone file that may exist if this volume has been already booted. This has been documented already in a few places. If you build images for deployment, you may have scenarios in which it’s important for the profile to be installed on the system’s first boot rather than later in its software management cycle.1

I got tired of fiddling with these details and making errors copying and pasting profile packaging/uninstall scripts from one profile to another, so I wrote a short Python utility to make this easier to automate. It’s called “make-profile-pkg”, and it lives here on GitHub. There are more details on the how and why there on the GitHub page.

Graham Gilbert contributed a couple great things on this: He helped make it a generic pkg-building tool rather than a Munki-specific tool, and also added the logic in the postinstall script for it to do the right thing about the install location depending on whether the package is used on a booted or non-booted volume. This allows a single package built with this utility to be installable on both booted and non-booted volumes. In addition to building the package with a few configurable options, it will also generate an uninstall script that can be used in conjunction with your software management platform of choice.

  1. An important distinction here is to set the package to use the full path starting with /private/var rather than simply /var, which is actually a symlink. It’s been reported to have caused issues in some cases before, and it’s simply not correct: install files to real paths rather than symlinks whenever possible.
Tagged , , | Leave a comment

Installing Command Line Tools automatically on Mavericks

In Mavericks, the Xcode Command Line Tools can be downloaded from the ADC downloads page like with previous versions. Now, though, they can be also be installed on-demand in a similar fashion to how Java has been installed since Lion, by simply invoking a command installed by them such as otool, or a new option in the xcode-select utility: --install.


In this post we’ll look at how you can trigger and run this installation in an automated way, eliminating the need for any user interaction.

Read More »

Tagged , , , , | 2 Responses