Category Archives: apple

Swift, Obj-C, AppleScript and AppStore fun.

How to access related files in the Mac Sandbox

I’m building an app which needs to load Objective C files.
If the user gives me access to the ‘.m’ file, I need to be able to open the ‘.h’ file.

You don’t get this ability without a bit of a dance. I didn’t find any clear tutorial or example, so I’m adding one now.

Firstly – define the related filetypes that you want to access by adding the file extension to document types in your info.plist
Here, I have added h as an an extension type, and set NSIsRelatedItemType to true.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>CFBundleDocumentTypes</key>
	<array>
        <dict>
            <key>CFBundleTypeExtensions</key>
            <array>
                <string>h</string>
            </array>
            <key>CFBundleTypeName</key>
            <string>Header File</string>
            <key>CFBundleTypeRole</key>
            <string>None</string>
            <key>NSIsRelatedItemType</key>
            <true/>
        </dict>
	</array>
</dict>
</plist>

Now, you can access the file – but only by using a FilePresenter.
And the file presenter _must_ be registered with NSFileCoordinator

class RelatedFilePresenter: NSObject, NSFilePresenter {
    var primaryPresentedItemURL: URL?
    var presentedItemURL: URL?
    var presentedItemOperationQueue: OperationQueue = OperationQueue.main
    
    enum Fail:Error {
        case unableToConstrucURL
    }
    
    init(primaryItemURL: URL) {
        self.primaryPresentedItemURL = primaryItemURL
    }

    func readDataFromRelated(fileExtension:String,
                             completion:(Result<Data, Error>)->Void) {
        presentedItemURL = primaryPresentedItemURL?.deletingPathExtension().appendingPathExtension(fileExtension)
        
        guard let url = presentedItemURL else {
            completion(.failure(Fail.unableToConstrucURL))
            return
        }
        
        let fc = NSFileCoordinator(filePresenter: self)
        var error:NSError?
        
        NSFileCoordinator.addFilePresenter(self)
               
        fc.coordinate(readingItemAt: url,
                      options: .withoutChanges,
                      error: &error) { (url) in
            do {
                let data = try Data(contentsOf: url, options: .uncached)
                completion(.success(data))
            } catch  {
                completion(.failure(error))
            }
        }
        
        if let error {
            completion(.failure(error))
        }
        
        NSFileCoordinator.removeFilePresenter(self)
    }
    
}

This class provides a method to read the data from a related file, but you could adapt it for any other operation.

You can then read the data with something like

let filePresenter = RelatedFilePresenter(primaryItemURL: mFileURL)
filePresenter.readDataFromRelated(fileExtension: "h") { result in
    if case .success(let data) = result {
        //do something with data
    }
}

Five ways to break Swift Concurrency

I really like Swift Concurrency.

I just wish Apple would clearly document what it actually guarantees

Generally, sites (including Apple) explain that marking a function with @MainActor guarantees that it will always run on main.

By adding the new @MainActor annotation to Photos, the compiler will guarantee that the properties and methods on Photos are only ever accessed from the main actor.

Discover Concurrency in SwiftUI – Apple

This simply isn’t true.

Here are five simple functions that cause a @MainActor function to run on a background thread…

class OnMain:NSObject {
    
    @MainActor
    @objc
    func noCanFail() {
        guard Thread.isMainThread else {
            fatalError("not on main")
        }
        print("Did Not Fail!")
    }

    //Just call the selector directly. Swift doesn't care
    func breakWithSelector() {
        Task {
            self.perform(#selector(self.noCanFail))
        }
    }
    
    // Get some Objective C code to call the 'protected' function
    // The objective C code is simply
    // [[OnMain new] noCanFail];
    func breakByCallingObjC() {
        Task {
            Trouble.cause()
        }
    }
    
    //NSNotification centre calls you back on whatever thread the notification was posted
    //It doesn't care about Swift Concurrency
    func breakByNotification() {
        let notification = Notification.Name("Cheeky")
        let center = NotificationCenter.default
        
        center.addObserver(self,
                           selector: #selector(self.noCanFail),
                           name: notification,
                           object: nil)
        Task {
            center.post(name: notification, object: nil)
        }
    }
    
    //Surprisingly, even with block syntax, you get the fatal error
    //At least there is a warning in this one
    func breakByNotification2() {
        let notification2 = Notification.Name("Cheeky2")
        
        let queue = OperationQueue()
        let center = NotificationCenter.default
        
        center.addObserver(forName: notification2,
                           object: nil,
                           queue: queue) { _ in
            self.noCanFail()
        }
        
        //No need to post from a task this time!
        center.post(name: notification2, object: nil)
    }
    
    
    //Objective C will just run the block for us by calling
    //block();
    //At least there is a warning in this one
    func breakWithBlock() {
        Task {
            Trouble.run {
                self.noCanFail()
            }
        }
    }

    func breakFoo() {
        let foo = Foo()
        let library = MyLibrary(delegate: foo)
        library.doWork()
    }
}

Two of these functions generate a warning in XCode 14.1 -three don’t.
They all trigger fatal errors when they end up calling noCanFail on a background thread.

This isn’t by any means an exclusive list.

The most obvious point where bugs like this can bite you is that any time you have a callback function from a system framework (such as bluetooth discovery, or even an Alert), or from a third party framework like a networking library – then it _could_ call back on any thread.

You might get lucky – but in this scenario, @MainActor does precisely nothing.

To my mind, the great sin here is not that Swift Concurrency has edge cases.

The great sin is that Swift Concurrency is presented as if it ‘just works’ – when in fact there are important caveats that you need to manage.

@MainActor is barely documented by Apple. There is a single oblique reference to it in ‘the Swift Programming Language’

It is a core part of the language, it’s ridiculous that Apple doesn’t have clear documentation on what it actually does.

The Rules (?)


My understanding is something like the following:

@MainActor works as long as you…

  1. Are calling from your own code
  2. Are not using selectors
  3. You only use Swift
  4. You don’t use any other form of concurrency (like OperationQueue)

NB – I am not claiming that these rules are 100% correct.
There are probably edge cases even within these rules.

Apple should document what the real rules are.

Rule #1 is trickier than you might think…

It is up to you to ensure that any code which is protected by @MainActor is only called directly from your own code.

If you have a callback from a library/framework, and that calls a function, which calls a function, which eventually calls your protected function – then that’s a problem.

This is true even if you’re providing a callback to an Apple framework like SKStoreKit

It’s _really_ easy to miss something like this.

Imagine the following common pattern

class Foo:NSViewController {

}

//perhaps conforming to some delegate protocol...
extension Foo:MyLibraryDelegate {
    func someCallback() {
        OnMain().noCanFail()
    }
}

You have a UIViewController or an NSViewController.

That does calls a framework/library (perhaps SKStoreKit or a networking library)

The delegate is able to call noCanFail() in the extension here because the whole class has inherited @MainActor from NSViewController

The library is something trivial that ends up calling the delegate off the main thread

protocol MyLibraryDelegate {
    func someCallback()
}

class MyLibrary {
    var delegate:MyLibraryDelegate
    
    init(delegate:MyLibraryDelegate){
        self.delegate = delegate
    }
    
    func doWork() {
        Task {
            delegate.someCallback()
        }
    }
}

Once again – a fatal error.
No warnings in the compiler (at least not with default XCode settings)

Swift Concurrency is great. I’ll continue to use it.

Carefully.

Update – Add flag for warnings…

Thomas Goyne pointed out that that you can add a build flag to generate warnings in these cases.
It doesn’t stop @MainActor methods from being called in the background – but it at least generates a purple runtime warning for you to investigate.



The flag is -Xfrontend -enable-actor-data-race-checks

Add it to ‘Other Swift Flags’

I’ll be adding this to all my projects.

Update – Keypaths…

Probably not surprising at this point. Keypaths seem to completely ignore Swift Concurrency

No Objective C involved here at all

    @MainActor
    var doNotFail:Int {
        guard Thread.isMainThread else {
            print("Failed")
            return 0
        }
        return 0
    }

    func breakWithKeypath() {    
        Task {
            var array = [OnMain()]
            var sorted = array.map(\.doNotFail)
        }
    }

doNotFail is merrily called on a background thread…

Fool me once… Reporting bugs to Apple

About a year ago, I decided to give up reporting bugs to Apple. Like many others, I got fed up of them simply being ignored.

At WWDC in in June of 2021, I signed up for a SwiftUI lab, and showed the two Apple engineers a bug that had been annoying me. The bug is small enough to fit in a tweet:

struct ContentView: View {
    
    @State var mode:Int = 0
    
    var body: some View {
        
        HStack(spacing:10) {
            
            Button("Button A") {}
 
            Button("Button B") {}
            
            Picker(selection: $mode, label: EmptyView()) {
                Text("1").tag(0)
                Text("2").tag(1)
            }
            .pickerStyle(SegmentedPickerStyle())
            
        }
        
    }
}

That code should produce two buttons and a segmented picker.

However, if you run it on a Mac in Catalyst, without the ‘optimise for Mac’ option, then as soon as you click on Button A – the picker becomes completely unresponsive.

We poked about a bit, confirmed that it really was a bug, and dug into the root cause (the button is leaving a gesture recogniser in an incorrect state)

There was no fix – but the pair told me that I should definitely submit a bug report.

When I explained that I had given up on bug reports – they assured me that their team didn’t ignore reports.
They love simple reproducible cases like this, and I should absolutely submit.

Fool me twice

I decided to give them the benefit of the doubt.

I went back to feedback reporter and found that I had actually reported this bug in December 2020. Nonetheless, I updated it to confirm that it was still broken on the latest OS and actually resubmitted under my company account with some discussion of the conversation I had at the lab.

The bug (FB8925084) includes the relevant code and a sample project.

The re-submission has never had any response at all.

My initial personal submission has the following history:

  • 9/Feb – I confirm that it is broken in Monterey beta
  • 2/July – Apple report that this is fixed in 11.3.1 (it isn’t)
  • 3/July – I confirm it is not fixed. Still broken in 11.4 and latest Monterey beta
  • 26/July – I confirm still broken in latest Monterey beta
  • 12/Aug – I confirm still broken in Monterey beta 5

What now?

SwiftUI & Catalyst are important platforms. This bug is remarkably simple to trigger.

It feels like the kind of thing that Apple should care about.

I certainly care about it – the underlying problem of bad gesture handling causes a bunch of issues in one of my apps.

Clearly reporting bugs is a waste of time though. I guess we just have to wait until Apple hit this in one of their own apps and have cause to care.

Update 31 Aug

Stil broken in Beta 6 (reported)

Update 4th Sept from Apple:


“Your feedback is important to us”

Update October 2021

It looks like Apple have finally addressed this in a Monterey beta.


I commented earlier:

I guess we just have to wait until Apple hit this in one of their own apps and have cause to care.’

And I suspect that is exactly what happened. The Shortcuts app in Monterey looks like it has been hitting this issue (or a related one).

I certainly don’t think my feedback helped to get things fixed. There has been no update to my report from Apple.

Dynamic Wallpaper Wierdness (again)

tldr; Multi Monitor Wallpaper can give you the daytime images for your dynamic wallpaper even if you use dark mode.

Dynamic Wallpaper

One of the trickier things that Multi Monitor Wallpaper does is pull apart dynamic wallpapers so that it can build separate ones for each of your monitors. It then puts them back together again.

These dynamic wallpapers should change with the sun – but they don’t always.

Sometimes, it is just one of Apple’s many frustrating bugs* around wallpapers.

Sometimes it is something that looks like it was intended as a feature.

One of those is the feature where dynamic wallpapers can declare which images are suitable for dark mode. Sometimes, the operating system will honour that declaration. When you switch to dark mode, even in the middle of the day, it will show you an evening/night image.

I thought Big Sur had stopped doing this, but it turns out the behaviour is just intermittent. On my iMac right now, the Catalina wallpaper shows based on the current time whatever your mode. The Big Sur wallpaper switches to match the mode.

On my Macbook air, they were both switching earlier today – but now neither is switching.

I have no idea what is causing this inconsistent behaviour. I have pulled out the metadata and they both have the same structure.

However – I can fix it so that you can see all the images whatever your mode.

Multi Monitor Wallpaper now has a setting (enabled by default) which simply tells the operating system that every image is suitable for use at night. This is based on the fantastic work by Marcin Czachurski.

You can turn this off if you prefer. In that case, you’ll get Apple’s default and unpredictable behaviour. It might show dark images when you use dark mode. It might not!


*for example the popup display bug reported when Mojave was in beta here, or the bug I have reported where disconnecting and re-connecting screens can corrupt the wallpaper settings database, or the bug where picking a still desktop means that wallpapers set via the API are also stuck in still mode. I could go on…

Apple Reviewers can Punish You

tl;dr – Apple policy may now say they won’t hold up your urgent bugfix, but that doesn’t stop them holding it ‘in review’ for an unusually long time so that it is never actually released…

I have had plenty of ‘robust discussions’ with App Review. Sometimes they come round to my way of thinking – I have had several appeals approved. Sometimes Apple just isn’t budging. Until this occasion though they have always been polite and engaged in good faith. I have never felt that a reviewer was punishing me out of spite. This time, I feel like I really got a ‘Bad Apple’ at Apple.

This particular Apple Review nightmare kicked off when a reviewer decided that my bugfix release:

  1. Enabled auto-launch without the user’s permission
  2. Had an app preview video which breaks the rules

Convincing them that they were wrong about #1 was easy. I just had to show the screenshot where my app asks the user if they want to enable the recommended settings and explicitly lists ‘auto-launch’ as one that will be enabled.

Number 2 was harder. Here is the video causing the problem:

For context, this video has been in place for about 18 months, and has (successfully) gone through about 30 reviews so far. You can see it now in the app store listing.

Multi Monitor Wallpaper is an app that sets the wallpaper across multiple monitors. It really needs a way in the app preview to show the multiple monitors in action. So – it ‘zooms out’ allowing you to see the full effect.

This, my reviewer insists breaches the (unwritten) rule

Your app preview includes content that does not sufficiently reflect the app in use. Specifically, your preview:

– Includes device images and/or device frames.

I understand where the frames thing comes from. Apple don’t want a video of people using their Mac in the coffee shop with the app running. They want to see the app itself.

In my case though – the monitors (and frames) _are_ the app. I think that makes this a reasonable and honest app preview. The reviewers when I first submitted the preview 18 months ago obviously agree, and none of the 30ish reviews since then have had a problem with it.

Nonetheless, my reviewer wasn’t budging. They said:

We advise appeal this review by sending a request to the App Review Board. As you may be aware, the App Review Board was created for developers to appeal an app review they have concerns with.
Once the App Review Board has completed their evaluation, they will contact you directly.

Recently, Apple published new guidelines which promised developers that

Bug Fix Submissions: For apps that are already on the App Store, bug fixes will no longer be delayed over guideline violations except for those related to legal issues.

https://developer.apple.com/news/?id=xqk627qu

I really like the new policy. It seems reasonable. This certainly was a bug fix submission. It fixes a crashing bug when users click on one of my image search providers (Unsplash)

I asked:

I did see your message suggesting that I should appeal to the app review board. I’m will do that today.

In the meantime, there is a significant bug which this version fixes (there is an immediate crash if you click to browse Unsplash).
As per your press release, I understand that ‘bug fixes will no longer be delayed over guideline violations except for those related to legal issues.’, so would appreciate it if you could publish this update as soon as possible.

And, I submitted my appeal, and copied it into the resolution centre to keep my reviewer up to date.

Hello Rob,

At this time, you will need to follow the pending guidance from the App Review Board.

I explained that I wasn’t aware of any pending guidance and asked again

Bug Fix Submissions: For apps that are already on the App Store, bug fixes will no longer be delayed over guideline violations except for those related to legal issues.[…]

this is a bug fix submission. I haven’t changed the AppPreview video. I would like to take advantage of this process.
I will address this issue in my next submission if my appeal is not supported by the Appeal Board

still ‘computer says no’

Hello Rob,

Thank you for your response.

To clarify, the App Review Board will be in contact with you, as you currently have a pending appeal.

Best regards,

I tried again

I’m happy for the review board to be in contact with me.
As you say – I have a pending appeal (at your explicit suggestion)

Apple have communicated publicly (via press release and direct email) that ‘bug fixes will no longer be delayed over guideline violations except for those related to legal issues.’

If this is indeed a guideline violation, it certainly doesn’t relate to legal issues.

I’m asking you to honour that explicit public commitment.

that got me close to what I needed

Thank you for providing this information.

If this is a bug fix submission, you may elect to have it approved at this time.

If you would like to take advantage of this opportunity, please respond in Resolution Center to confirm that you can address these issues in your next submission. Additionally, please let us know if you have any questions about how to resolve these issues. If you believe this decision was made incorrectly, you may appeal your app rejection. You may also provide feedback on our review guidelines.

well – I must admit I thought the reviewer was just throwing up an extra step here to be annoying, but I confirmed. Shortly later, I got the promising response

Thank you for providing this information.

We will continue the review, and we will notify you if there are any further issues.


Now. We need a little diversion. How long does a review take?
I have put quite a few updates on Multi Monitor Wallpaper.

This is how long the recent reviews took from the point where they moved to the ‘In Review’ status.

  • 9-Sept, 5 mins
  • 8 Sept, 16 mins
  • 25 July, 20mins
  • 23 July, 20 mins
  • 21 July, 24 mins
  • 16 July, 7 hrs
  • 24 Jun, 1hr 40mins
  • 11 Jun, 6hrs

Mostly about 20 mins. Sometimes 6 or 7 hours.


How long do you think it takes for a very simple bugfix with no changes to the overall app – where Apple is allowing it through because of their commitment ‘not to hold up bugfix releases’ ?

I don’t know the answer to that.

My app changed status to ‘In Review’ on the 11 Sep 2020 at 23:27

It has been over 60 hours so far and I am still waiting.

After 24 hours, I even submitted a request for an expedited review. That was granted – but nothing has happened yet.

Perhaps I have just been unlucky, and the reviewer who seemed to deliberately throw up steps to avoid allowing my bugfix until I asked whether Apple were good to their word; Perhaps that reviewer just had a family emergency and accidentally left my app in the queue.

I think they’re punishing me.

They’re also punishing our shared customers who currently have to use a crashing app.

I’ll keep this post updated.


Update. 68 hours after moving to ‘In Review’, Multi Monitor Wallpaper was finally approved.

Remember that this was the process to ‘not hold up an urgent bugfix release’


Note – quotes of discussions with App Review are fairly heavily edited for brevity, in that they don’t show everything said. Everything quoted is verbatim though.

If any press want full details, I’ll be happy to share the complete conversation.

Google/Apple. Requiring the Account Holder to do techie things is BAD SECURITY!

tl;dr – in the name of security, you encourage me to share passwords.

Photo by Jon Moore on Unsplash

Both the Apple App Store, and the Google Play store allow business accounts to have multiple users.

This means that my client (the app owner) can own their account, and they can let me (the developer) do what I need to to develop and publish apps.

This means we don’t have to share passwords, and the account holder can limit what I have access to. So, for example I might be able to update apps – but I can’t see their financial reports. This all makes lots of sense.

Unfortunately, both Apple and Google have technical actions where they require the account holder to perform them. Because the actions are pretty technical, if the account holder is a business owner rather than a techie – they probably don’t have the skills to perform them. I hit these both recently.

In both cases, they’re important security-sensitive actions. For Apple – it was creating Developer ID Certificates to let me upload a MacOS Catalyst app to the store. For Google, it was enabling API access to automate uploading new builds.

Both of these actions are absurdly technical for a non-tech person to complete. Follow the links if you want the gory details!*

This means that the only practical way to perform these necessary actions (if you’re not physically located in the same place) is for the account owner to share their password with me the developer. This is clearly terrible security practice – and exactly what the multi-user system is set up to avoid.

I completely understand that these are security-critical steps. It makes sense that the account holder should have some kind of approval when they happen. This could be an explicit post-action approval:

‘Rob has requested XXXX – this is a critical security issue. Do you want to approve this action’

Or it could be a time-limited user permission:

‘Grant Rob permission to do XXX for the next 24 hours’

The current system achieves the opposite of what it sets out to do. It sets out to keep security-critical actions safe, but what it encourages is that the developer probably gets the account holder’s password and complete access to everything.


*for bonus foolishness – in the case of the Apple action. Xcode will automatically generate your certificates if you add the account holder’s account to Xcode. This means that it is complicated and techie for the business person to perform the actions (they don’t have Xcode installed) – but automated for the developer if they can just get the account holder’s password.

Dolby Codecs, Legal Threats: Coda

Back in 2014, I wrote about how Dolby were requiring me to remove ac3 support from VLC Streamer for somewhat questionable legal reasons

I included the email I received from their representative about the price I would have to pay

You should also know that there are fees associated with becoming licensed.  Specifically, there is a $25,000 one-time initial fee and a recurring $5,000 annual maintenance fee.  There is also a per-unit royalty that has a tiered structure, due quarterly, based on annual total usage, as follows:

0-100,000 downloads at $0.99 per download

100,001-1,000,000 downloads at $0.67 per download

1,000,001+ downloads at $0.45 per download

I was interested to get a call from a lawyer working for Adobe recently. They were being sued by Dolby for something related to these patents.

Bizarrely, Dolby were arguing that their pricing levels were so super secret that they couldn’t even disclose them in court.

Adobe found my old post and argued that if they were published on the web – they couldn’t be that secret.

I don’t have any details about the case, but I know they settled and it went away. Hopefully my pricing info made the difference…