{ "version": "https://jsonfeed.org/version/1", "title": "Joseph Duffy", "home_page_url": "https://josephduffy.co.uk/posts", "feed_url": "https://josephduffy.co.uk/feed.json", "description": "Blog posts written by Joseph Duffy", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" }, "items": [ { "id": "https://josephduffy.co.uk/posts/the-bug-that-bit-me-twice", "content_html": "
I've been working on a fix for a bug in Overamped, which causes the popover UI shown when tapping on an image in Google Images to be blank, if the link goes to an AMP page. This was a silly bug that never should've happened; knowing that Google can change their page structure at any time I should've been more cautious with my checks.
\nAs a temporary quick fix I removed all custom handling of Google results, tested my changes in the simulator, and uploaded a new build to TestFlight.
\nAfter installing the TestFlight update on my phone I checked a search result that I knew recreated the problem, but it was still happening! I have other extensions installed so I disabled some, refreshed, and the bug was fixed!
\nI thought it would be very strange for the same – very specific – bug to appear in multiple extensions, so I did a little digging.
\nI quickly narrowed down the extension that was causing this bug. Knowing that the same extension is installed on my Mac – and that if I were to release an app on multiple platforms I would use the same JavaScript across platforms – I inspected the app and found the extension. Inside was a file that looked like exactly what I was looking for: redirect-amp.js
.
After opening and formatting the file for easier reading, I took a brief look over it. Something stood out to me as soon as I spotted it:
\nconst c = (function (e) {\n const t = e.querySelector("span[aria-label='AMP logo']")\n if (t) return t\n if (e.dataset.ampHlt) {\n let t = e.parentElement\n for (; t && !t.classList.contains("card-section"); ) t = t.parentElement\n if (t) return t.querySelector("span[aria-label='AMP logo']")\n }\n return null\n})(e)\n
\nThis felt incredibly familiar, so I compared it with my own code:
\nfunction findAMPLogoRelativeToAnchor(\n anchor: HTMLAnchorElement,\n): HTMLSpanElement | null {\n const childLogo = anchor.querySelector("span[aria-label='AMP logo']")\n\n if (childLogo) {\n return childLogo as HTMLSpanElement\n }\n\n if (anchor.dataset.ampHlt) {\n console.debug(\n `Anchor is from a "Featured Snippet"; searching parent for container`,\n )\n // The "Featured Snippet" puts the logo outside of the anchor\n let parent = anchor.parentElement\n\n while (parent && !parent.classList.contains("card-section")) {\n parent = parent.parentElement\n }\n\n if (parent) {\n console.debug("Found card section parent", parent)\n return parent.querySelector(\n "span[aria-label='AMP logo']",\n ) as HTMLSpanElement | null\n }\n }\n\n console.debug("Failed to find corresponding AMP logo <span> for", anchor)\n\n return null\n}\n
\nOk, so maybe that's a just a coincidence; there are only so many ways to do these lookups. However, it's worth noting that currently, to the best of my knowledge, this code does nothing. When I started working on my AMP redirector in June 2021 Google would show a small bolt next to AMP results. The aim of this code was to remove that bolt. However, before the app was released in September 2021 Google had removed this bolt, so this code was effectively useless. I left it in just in case they decided to re-add the bolt in the future.
\nAt this point I was suspicious, so I bug a little deeper so something more specific: the code causing the bug.
\nMy code would remove anything it thought was an AMP popover, but the search for the popover wasn't cautious enough:
\ninterface AnchorAttributes {\n url: string\n ampPopover: Element | null\n}\n\n// The URL to redirect to – if found – and the element\n// that contains the AMP popover (used in image searches)\nconst attributes = ((): AnchorAttributes | null => {\n const ampCur = anchor.dataset.ampCur\n\n if (ampCur && ampCur.length > 0) {\n // data-amp-cur is available on News search results (not news.google)\n // and has the full canonical URL\n hasCanonicalURL = true\n return { url: ampCur, ampPopover: null }\n }\n\n if (anchor.dataset.amp) {\n return { url: anchor.dataset.amp, ampPopover: null }\n }\n\n if (anchor.dataset.cur) {\n return { url: anchor.dataset.cur, ampPopover: null }\n } else {\n // Check if this is an AMP result within an image search result\n // This is a little fragile but seems to be the most efficient\n // without replacing links that aren't to AMP pages.\n //\n // TODO: Check if it's possible to detect links on image search\n // result pages. e.g. the Universe Today link on\n // https://www.google.co.uk/search?q=eta+carinae&client=safari&hl=en-gb&prmd=nivx&source=lnms&tbm=isch&sa=X&ved=2ahUKEwjBqdOumez1AhXoJEQIHS7UBWAQ_AUoAnoECAIQAg&biw=375&bih=635&dpr=3\n\n // This is a div containing the links\n const upperContainer = anchor.parentElement?.parentElement\n\n if (!upperContainer) {\n return null\n }\n\n if (!upperContainer.nextElementSibling) {\n // Likely not an AMP link; the next sibling should be\n // the element that displays the AMP page\n return null\n }\n\n // Double check this is in fact an AMP link\n if (\n upperContainer.nextElementSibling.querySelector(\n "div[aria-label*='AMP']",\n ) === null\n ) {\n return null\n }\n\n return { url: anchor.href, ampPopover: upperContainer.nextElementSibling }\n }\n})()\n\nconst { url: anchorURLString, ampPopover } = attributes\n\nconst anchorURL = new URL(anchorURLString)\n\nif (anchorURL.hostname === window.location.hostname) {\n // Do not override internal links, e.g. links to `"#"` used for anchors acting as buttons\n // `role="button"` could also be used but may exclude too many anchors\n return\n}\n\n// ...\n\nif (ampPopover) {\n console.debug("Removing AMP popover", ampPopover)\n ampPopover.remove()\n}\n
\nOh how innocent the "Double check this is in fact an AMP link" comment seems, now knowing the bug it didn't prevent 😅 (the bug is that it's only checking if it contains a div
with aria-label
contains "AMP", which would technically be true for any parent element)
Searching in the same extension's code reveals the following:
\nconst [r, o] = (() => {\n var t\n const { ampCur: r } = e.dataset\n if (r && r.length > 0) return [r, null]\n if (e.dataset.amp) return [e.dataset.amp, null]\n if (e.dataset.cur) return [e.dataset.cur, null]\n const n =\n null === (t = e.parentElement) || void 0 === t ? void 0 : t.parentElement\n return (null == n ? void 0 : n.nextElementSibling) &&\n null !== n.nextElementSibling.querySelector("div[aria-label*='AMP']")\n ? [e.href, n.nextElementSibling]\n : e.href && new URL(e.href).hostname !== location.hostname\n ? [e.href, null]\n : [null, null]\n})()\n\n// ...\n\no && o.remove()\n
\nAt this point I was quite confident: somehow, my code had ended up in this extension.
\nAs a user of the app in question for a long time (my emails show I purchased it in May 2016 and have been subscribed since January 2020) I assumed either I was mistaken and this was all a coincidence, this was an honest mistake, or something like GitHub Copilot had chewed up my code and spat it out for the extension's development team.
\nHoping to get some insight on this I tweeted some screenshots of the code and mentioned the developer of the extension, fully expecting the tweet to enter the void and for me to hear nothing more of it.
\nTo my surprise I woke up the next day to a Twitter DM explaining that the the extension developer hired someone else to write this part of the extension, they didn't know this code had been copied, and that they had removed the offending code and submitted an update to Apple 🎉 All of this, easily within 12 hours of my finding the offending code.
\nYou may have noticed I didn't mention the extension in question in this post. That's because the point of this was tell the story of how I found the copied code, and also because I find it funny that a bug of my own creation came back to bite me twice: once in my own app, and once in a different one!
\nMy original tweet does mention the app that included this copied code, but all I will say is that I am and plan to continue being a customer of this app, and if there's anything else you take away from this post it should be that the developers behind it were not acting maliciously. I would, however, like to know more about the developer that submitted my work to them though 😈
\nFor those wondering: the Overamped source code is available on GitHub, which is probably where this code was copied from. The source code is available for auditing purposes only and is not licensed. As such, the code in this post does not fall under the usual CC-BY-4.0 license used by other posts on this website.
\nI did expect something like this to happen eventually, but I always assumed it would be more brazen – such as a clear re-upload of the app using a different name – and didn't expect it to be resolved so quickly.
", "url": "https://josephduffy.co.uk/posts/the-bug-that-bit-me-twice", "title": "The Bug That Bit Me Twice", "summary": "I've been working on a fix for a bug in Overamped, which causes the popover UI shown when tapping on an image in Google Images to be blank, if the link goes to an AMP page. This was a silly bug that never should've happened; knowing that Google can change their page structure at any time I should've been more cautious with my checks.
\nAs a temporary quick fix I removed all custom handling of Google results, tested my changes in the simulator, and uploaded a new build to TestFlight.
\nAfter installing the TestFlight update on my phone I checked a search result that I knew recreated the problem, but it was still happening! I have other extensions installed so I disabled some, refreshed, and the bug was fixed!
\nI thought it would be very strange for the same – very specific – bug to appear in multiple extensions, so I did a little digging.
", "date_modified": "2022-12-01T19:45:37.000Z", "date_published": "2022-12-01T19:45:37.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/partial-in-swift", "content_html": "\nPartial is now available in its own Swift package on GitHub. This post is still valid, but somewhat out of date.\n
\nStructs are incredibly useful in Swift, especially when representing static read-only data. However, the values of a struct often come from multiple sources, such as view controllers, network requests, and files on disk, which can make the creation of these structs cumbersome.
\nThere are numerous methods to work around this, but each have their downsides. One of these methods is to change the struct to a class and update the properties to vars, but this removes the advantages of read-only structs. Another is to make a "builder" object, but the API of this object must be kept in-sync with the object is wraps.
\nPartial
eliminates these problems by providing a type-safe API for building structs by utilising generics and KeyPath
s. Although I learned of the concept of Partial
through TypeScript – which provides Partial
as a built-in type – the Swift implementation supports many more use cases.
To demonstrate the use of Partial
I will use some simple structs with a few let properties.
struct Order {\n\n let userId: Int\n\n let itemIds: [Int]\n\n let promoCode: String?\n\n let address: Address\n\n let billingDetails: BillingDetails\n\n}\n\nstruct Address {\n\n let name: String\n\n let firstLine: String\n\n let additionalLines: [String]\n\n let city: String\n\n let postCode: String\n\n}\n\nstruct BillingDetails {\n\n let creditCardNumber: String\n\n let ccv: String\n\n let address: Address\n\n}\n
\nFor simple use cases, only a very simple implementation is required.
\nstruct Partial<Wrapped> {\n\n private var values: [PartialKeyPath<Wrapped>: Any] = [:]\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType>) -> ValueType? {\n get {\n return values[key] as? ValueType\n }\n set {\n values[key] = newValue\n }\n }\n\n}\n
\nYou can get and set values by using the subscript of a Partial
and passing a KeyPath
of the wrapped type.
var partialOrder = Partial<Order>()\npartialOrder[\\.userId] // nil\npartialOrder[\\.userId] = 123\npartialOrder[\\.userId] // 123\n
\nHowever, Partial
s can be much for more powerful than just this.
One of the first issues you run across with Partial
s as basic as this is that the initialisation of the wrapped type can be a bit cumbersome.
struct Order {\n\n // ...\n\n init?(partial: Partial<Order>) {\n guard let userId = partial[\\.userId] else { return nil }\n guard let itemIds = partial[\\.itemIds] else { return nil }\n\n self.userId = userId\n self.itemIds = itemIds\n self.promoCode = partial[\\.promoCode]\n\n // Must check for both scenarios\n let partialAddress = partial[partial: \\Order.address]\n if let name = partialAddress[\\.name],\n let firstLine = partialAddress[\\.firstLine],\n let additionalLines = partialAddress[\\.additionalLines],\n let city = partialAddress[\\.city],\n let postCode = partialAddress[\\.postCode] {\n self.address = Address(name: name, firstLine: firstLine, additionalLines: additionalLines, city: city, postCode: postCode)\n } else if let address = partial[\\.address] {\n self.address = address\n } else {\n return nil\n }\n\n // Same must be done for `billingDetails`...\n }\n\n
\nBy defining a new protocol, a throwing function that can retrieve values, and adding a new subscript that can "unwrap" any values stored in sub-Partial
s, the call site can be much more concise and clear.
protocol PartialConvertible {\n\n init(partial: Partial<Self>) throws\n\n}\n\nstruct Partial<Wrapped> {\n\n // ...\n\n enum Error: Swift.Error {\n case missingKey(PartialKeyPath<Wrapped>)\n case invalidValueType(key: PartialKeyPath<Wrapped>, actualValue: Any)\n }\n\n func value<ValueType>(for key: KeyPath<Wrapped, ValueType>) throws -> ValueType {\n if let value = values[key] {\n if let value = value as? ValueType {\n return value\n } else {\n throw Error.invalidValueType(key: key, actualValue: value)\n }\n } else if let value = backingValue?[keyPath: key] {\n return value\n } else {\n return Error.missingKey(key)\n }\n }\n\n func value<ValueType>(for key: KeyPath<Wrapped, ValueType>) throws -> ValueType where ValueType: PartialConvertible {\n if let value = values[key] {\n if let value = value as? ValueType {\n return value\n } else if let partial = value as? Partial<ValueType> {\n return try ValueType(partial: partial)\n } else {\n throw Error.invalidValueType(key: key, actualValue: value)\n }\n } else if let value = backingValue?[keyPath: key] {\n return value\n } else {\n throw Error.missingKey(key)\n }\n }\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType>) -> ValueType? where ValueType: PartialConvertible {\n get {\n return try? value(for: key)\n }\n set {\n values[key] = newValue\n }\n }\n\n}\n\nstruct ExportOptions: PartialConvertible {\n\n // ...\n\n init(partial: Partial<Order>) throws {\n userId = try partial.value(for: \\.userId)\n itemIds = try partial.value(for: \\.itemIds)\n promoCode = try partial.value(for: \\.promoCode)\n address = try partial.value(for: \\.address)\n billingDetails = try partial.value(for: \\.billingDetails)\n }\n\n}\n\nextension Order.Address: PartialConvertible {\n\n init(partial: Partial<Order.Address>) throws {\n name = try partial.value(for: \\.name)\n firstLine = try partial.value(for: \\.firstLine)\n additionalLines = try partial.value(for: \\.additionalLines)\n city = try partial.value(for: \\.city)\n postCode = try partial.value(for: \\.postCode)\n }\n\n}\n\nextension Order.BillingDetails: PartialConvertible {\n\n init(partial: Partial<Order.BillingDetails>) throws {\n creditCardNumber = try partial.value(for: \\.creditCardNumber)\n ccv = try partial.value(for: \\.ccv)\n address = try partial.value(for: \\.address)\n }\n\n}\n
\nMuch better!
\nPartial
sPartial
s on their own are great, but once you try to access a property of a property of a Partial
it stops working quite as expected.
partialOrder[\\.address][\\.name] = "Santa Claus" // Not possible\n
\nTo support this a new subscript and a value(for:)
function that utilises the PartialConvertible
protocol is required.
\nstruct Partial<Wrapped> {\n\n // ...\n\n func value<ValueType>(for key: KeyPath<Wrapped, ValueType>) throws -> ValueType where ValueType: PartialConvertible {\n guard let value = values[key] else {\n throw Error.missingKey(key)\n }\n if let value = value as? ValueType {\n return value\n } else if let partial = value as? Partial<ValueType> {\n return try ValueType(partial: partial)\n } else {\n throw Error.invalidValueType(key: key, actualValue: value)\n }\n }\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType>) -> Partial<ValueType> where ValueType: PartialConvertible {\n get {\n return values[key] as? Partial<ValueType> ?? Partial<ValueType>()\n }\n set {\n values[key] = newValue\n }\n }\n\n}\n\npartialOrder[\\.address][\\.name] // nil\npartialOrder[\\.address][\\.name] = "Johnny Appleseed"\npartialOrder[\\.address][\\.name] // "Johnny Appleseed"\n
\nHowever, because it will always return a Partial
, there will be an issue if the value has been explicitly set elsewhere:
partialOrder[\\.address] = Address(name: "Johnny Appleseed", ...)\npartialOrder[\\.address] // An empty `Partial`\ntry? Order.Address(partial: partialOrder[\\.address]) // nil\n
\nTo support this a backing value is added, allowing the stored value to be wrapped and its properties overridden.
\nThe type of the values
property is also updated to [PartialKeyPath<Wrapped>: Any?]
and subscript setters are updated to use the updateValue(_:forKey:)
function. This is to support unsetting values by assigning a key to nil
when a backing value is used.
struct Partial<Wrapped> {\n\n // ...\n\n private var values: [PartialKeyPath<Wrapped>: Any?] = [:]\n\n private var backingValue: Wrapped? = nil\n\n init(backingValue: Wrapped? = nil) {\n self.backingValue = backingValue\n }\n\n func value<ValueType>(for key: KeyPath<Wrapped, ValueType>) throws -> ValueType {\n if let value = values[key] {\n if let value = value as? ValueType {\n return value\n } else {\n throw Error.invalidValueType(key: key, actualValue: value)\n }\n } else if let value = backingValue?[keyPath: key] {\n return value\n } else {\n throw Error.missingKey(key)\n }\n }\n\n func value<ValueType>(for key: KeyPath<Wrapped, ValueType>) throws -> ValueType where ValueType: PartialConvertible {\n if let value = values[key] {\n if let value = value as? ValueType {\n return value\n } else if let partial = value as? Partial<ValueType> {\n return try ValueType(partial: partial)\n } else {\n throw Error.invalidValueType(key: key, actualValue: value)\n }\n } else if let value = backingValue?[keyPath: key] {\n return value\n } else {\n throw Error.missingKey(key)\n }\n }\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType>) -> ValueType? {\n get {\n return try? value(for: key)\n }\n set {\n values.updateValue(newValue, forKey: key)\n }\n }\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType>) -> Partial<ValueType> where ValueType: PartialConvertible {\n get {\n if let value = try? self.value(for: key) {\n return Partial<ValueType>(backingValue: value)\n } else if let partial = values[key] as? Partial<ValueType> {\n return partial\n } else {\n return Partial<ValueType>()\n }\n }\n set {\n values.updateValue(newValue, forKey: key)\n }\n }\n\n}\n\npartialOrder[\\.address] = Address(name: "Mr Appleseed", ...)\npartialOrder[\\.address][\\.name] // "Mr Appleseed"\npartialOrder[\\.billingAddress][\\.address] = partialOrder[\\.address]\npartialOrder[\\.billingAddress][\\.address][\\.name] = "Johnny Appleseed"\npartialOrder[\\.billingAddress][\\.address][\\.name] // "Johnny Appleseed"\n
\nWhen using a property of Wrapped
that's optional, such as promoCode
on Order
, the type of partial[\\.promoCode]
will be String??
. To work around this every function and subscript needs to be duplicated to support a key of type KeyPath<Wrapped, ValueType?>
.
For the sake of brevity, only one of these is shown below.
\nstruct Partial<Wrapped> {\n\n // ...\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType?>) -> ValueType? {\n get {\n return try? value(for: key)\n }\n set {\n values.updateValue(newValue, forKey: key)\n }\n }\n\n}\n
\nSwift will pick the right one based on context.
\nUsing Partial
does have some downsides. One is that you still have to create a custom init
function, a requirement that could be removed by adding Partial
to the standard library or made easier using metaprogramming tools such as Sourcery.
Another downside is that Xcode will not provide autocomplete suggestions for KeyPath
s, unless the type is provided before the period.
partialOrder[\\.userId] // Will not autocomplete\npartialOrder[\\Order.userId] // Will autocomplete\n
\nThis is not an issue with Partial
itself but is a shortcoming due to its reliance on KeyPath
Partial
is still a value type, which prevents the same instance being passed between objects. Some may choose to update Partial
to be a class, but I prefer to provide a small wrapper in the form of a class with a single partial
property and a convenient function for PartialConvertible
values. It could be subclassed or extended to support per-type convenience functions, a delegate, a completion closure, etc.
class PartialBuilder<Wrapped> {\n\n var partial: Partial<Wrapped>\n\n init(partial: Partial<Wrapped> = Partial<Wrapped>()) {\n self.partial = partial\n }\n\n init(backingValue: Wrapped) {\n partial = Partial(backingValue: backingValue)\n }\n\n}\n\nextension PartialBuilder where Wrapped: PartialConvertible {\n\n func unwrappedValue() throws -> Wrapped {\n return try Wrapped(partial: partial)\n }\n\n}\n\n
\nPartial
is available as part of the Partial
Swift package on GitHub.
Below is the full code – excluding documentation and CustomStringConvertible
and CustomDebugStringConvertible
conformance for the sake of brevity – plus a full example of how Partial
can be used.
struct Partial<Wrapped>: CustomStringConvertible, CustomDebugStringConvertible {\n\n enum Error<ValueType>: Swift.Error {\n case missingKey(KeyPath<Wrapped, ValueType>)\n case invalidValueType(key: KeyPath<Wrapped, ValueType>, actualValue: Any)\n }\n\n private var values: [PartialKeyPath<Wrapped>: Any?] = [:]\n\n private var backingValue: Wrapped? = nil}\n }\n\n init(backingValue: Wrapped? = nil) {\n self.backingValue = backingValue\n }\n\n func value<ValueType>(for key: KeyPath<Wrapped, ValueType>) throws -> ValueType {\n if let value = values[key] {\n if let value = value as? ValueType {\n return value\n } else if let value = value {\n throw Error.invalidValueType(key: key, actualValue: value)\n }\n } else if let value = backingValue?[keyPath: key] {\n return value\n }\n\n throw Error.missingKey(key)\n }\n\n func value<ValueType>(for key: KeyPath<Wrapped, ValueType?>) throws -> ValueType {\n if let value = values[key] {\n if let value = value as? ValueType {\n return value\n } else if let value = value {\n throw Error.invalidValueType(key: key, actualValue: value)\n }\n } else if let value = backingValue?[keyPath: key] {\n return value\n }\n\n throw Error.missingKey(key)\n }\n\n func value<ValueType>(for key: KeyPath<Wrapped, ValueType>) throws -> ValueType where ValueType: PartialConvertible {\n if let value = values[key] {\n if let value = value as? ValueType {\n return value\n } else if let partial = value as? Partial<ValueType> {\n return try ValueType(partial: partial)\n } else if let value = value {\n throw Error.invalidValueType(key: key, actualValue: value)\n }\n } else if let value = backingValue?[keyPath: key] {\n return value\n }\n\n throw Error.missingKey(key)\n }\n\n func value<ValueType>(for key: KeyPath<Wrapped, ValueType?>) throws -> ValueType where ValueType: PartialConvertible {\n if let value = values[key] {\n if let value = value as? ValueType {\n return value\n } else if let partial = value as? Partial<ValueType> {\n return try ValueType(partial: partial)\n } else if let value = value {\n throw Error.invalidValueType(key: key, actualValue: value)\n }\n } else if let value = backingValue?[keyPath: key] {\n return value\n }\n\n throw Error.missingKey(key)\n }\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType>) -> ValueType? {\n get {\n return try? value(for: key)\n }\n set {\n values.updateValue(newValue, forKey: key)\n }\n }\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType?>) -> ValueType? {\n get {\n return try? value(for: key)\n }\n set {\n values.updateValue(newValue, forKey: key)\n }\n }\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType>) -> Partial<ValueType> where ValueType: PartialConvertible {\n get {\n if let value = try? self.value(for: key) {\n return Partial<ValueType>(backingValue: value)\n } else if let partial = values[key] as? Partial<ValueType> {\n return partial\n } else {\n return Partial<ValueType>()\n }\n }\n set {\n values.updateValue(newValue, forKey: key)\n }\n }\n\n subscript<ValueType>(key: KeyPath<Wrapped, ValueType?>) -> Partial<ValueType> where ValueType: PartialConvertible {\n get {\n if let value = try? self.value(for: key) {\n return Partial<ValueType>(backingValue: value)\n } else if let partial = values[key] as? Partial<ValueType> {\n return partial\n } else {\n return Partial<ValueType>()\n }\n }\n set {\n values.updateValue(newValue, forKey: key)\n }\n }\n\n}\n\nvar partialOrder = Partial<Order>()\npartialOrder[\\.userId] = 123\npartialOrder[\\.itemIds] = [1, 4, 7]\npartialOrder[\\.promoCode] = "HELLO10"\npartialOrder[\\.address] = Address(name: "Johnny Appleseed", firstLine: "One Infinite Loop", additionalLines: ["Cupertino"], city: "CA", postCode: "95014")\npartialOrder[\\.billingDetails][\\.creditCardNumber] = "1111 2222 3333 4444"\npartialOrder[\\.billingDetails][\\.ccv] = "123"\npartialOrder[\\.billingDetails][\\.address] = partialOrder[\\.address]\npartialOrder[\\.billingDetails][\\.address][\\.name] = "Santa Claus"\npartialOrder[\\.billingDetails][\\.address][\\.firstLine] = "Santa's Grotto"\npartialOrder[\\.billingDetails][\\.address][\\.additionalLines] = []\npartialOrder[\\.billingDetails][\\.address][\\.city] = "Reindeerland"\npartialOrder[\\.billingDetails][\\.address][\\.postCode] = "XM4 5HQ"\n\ndo {\n let order = try Order(partial: partialOrder)\n order.userId // 123\n order.address.name // Johnny Appleseed\n order.billingDetails.address.name // "Santa Clause"\n} catch {\n error\n}\n
\nShaps helped me a lot with this post, from working with me through the evolution of the implementation to reading drafts of this post. Thanks, Shaps!
", "url": "https://josephduffy.co.uk/posts/partial-in-swift", "title": "Partial in Swift", "summary": "\nPartial is now available in its own Swift package on GitHub. This post is still valid, but somewhat out of date.\n
\nStructs are incredibly useful in Swift, especially when representing static read-only data. However, the values of a struct often come from multiple sources, such as view controllers, network requests, and files on disk, which can make the creation of these structs cumbersome.
\nThere are numerous methods to work around this, but each have their downsides. One of these methods is to change the struct to a class and update the properties to vars, but this removes the advantages of read-only structs. Another is to make a "builder" object, but the API of this object must be kept in-sync with the object is wraps.
\nPartial
eliminates these problems by providing a type-safe API for building structs by utilising generics and KeyPath
s. Although I learned of the concept of Partial
through TypeScript – which [provides Partial
as a built-in type][1] – the Swift implementation supports many more use cases.
Extending types in Swift support setting the scope for the extension, i.e. public
, internal
, or private
, with internal
being implicit if nothing is specified.
This may seem useful, but given the following snippet it's impossible to know what the scope of a function is:
\nfunc doSomething() {\n // Do the thing\n}\n
\nThe reason for this is that the context is entirely lost. If there's a chance this is declared on an extension it could be literally anything. Without having all the surrounding context the line above – or hundreds of lines above – there could be an extension MyType
with any scope.
It could be any of these:
\n/* Implicitly internal */ extension MyType {\n func doSomething() {\n // Do the thing\n }\n}\n
\npublic extension MyType {\n func doSomething() {\n // Do the thing\n }\n}\n
\ninternal extension MyType {\n func doSomething() {\n // Do the thing\n }\n}\n
\nprivate extension MyType {\n func doSomething() {\n // Do the thing\n }\n}\n
\nI've been aiming to improve the maintainability of my code lately, and this is one of the things I'm doing to improve that. If I want to jump in and quickly fix a bug or make a small update I don't want to guess the access level of anything. Making the access level explicit and never setting an access level on an extension helps with this.
\npublic func doSomething() {\n // Do the thing\n}\n
\nIn frameworks I've found it useful always be explicit about the access level. This forces me to think about a symbol being public
or internal
, so even if the internal
is implicit I like to add it. Maybe I'll change my mind of that though, just as I did with explicit self
😬
Extending types in Swift support setting the scope for the extension, i.e. public
, internal
, or private
, with internal
being implicit if nothing is specified.
This may seem useful, but given the following snippet it's impossible to know what the scope of a function is:
\nfunc doSomething() {\n // Do the thing\n}\n
",
"date_modified": "2022-03-21T21:13:34.000Z",
"date_published": "2022-03-21T21:13:34.000Z",
"author": {
"name": "Joseph Duffy",
"url": "https://josephduffy.co.uk/"
}
},
{
"id": "https://josephduffy.co.uk/posts/overamped-1-1-0-year-of-small",
"content_html": "So far my Year of Small is going really well; as I write this I'm making some final changes and preparing to push out v1.1.0-RC.1, which I hope to submit the App Store in the next couple of days.
\nOveramped 1.1.0 is an update I first started working on almost 3 months ago, and it should've been released earlier.
\nMy initial plan for 1.1.0 was to:
\nEasy, right? Well, adding widgets is why this update has been so delayed.
\nI initially decided to add widgets because I wanted to improve my design skills and thought that the constrained environment would help me, but that never worked out. At first I wanted to use the vibrant background found in Apple's widgets. I soon found out this isn't possible and really struggled with the design. I started with barebones widgets and thought I'd try them out on my homescreen. They've been sitting there for months and whenever I see them I don't like them. Because of this I was demotivated to work on it further, until I started my Year of Small.
\nPart of my focus for my Year of Small is to release a higher quantity of smaller updates. With this mindset I decided to remove the widgets from the app (they're still in the project, but disabled) and release the update without it.
\nI'm really hopeful that my Year of Small will be successful, and I'm already excited to work on other small updates and a small utility macOS app. All posts relating to my Year of Small can be found under the year-of-small tag.
", "url": "https://josephduffy.co.uk/posts/overamped-1-1-0-year-of-small", "title": "Overamped 1.1.0 and the Year of Small", "summary": "So far my Year of Small is going really well; as I write this I'm making some final changes and preparing to push out v1.1.0-RC.1, which I hope to submit the App Store in the next couple of days.
\nOveramped 1.1.0 is an update I first started working on almost 3 months ago, and it should've been released earlier.
\nMy initial plan for 1.1.0 was to:
\nEasy, right? Well, adding widgets is why this update has been so delayed.
", "date_modified": "2022-01-29T08:08:16.000Z", "date_published": "2022-01-29T08:08:16.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/pastelghouls-available-again", "content_html": "pastelghouls is once again available for download on the App Store. It's a free sticker pack containing 6 ghoulish sticker. Originally released 19th October 2016, just in time for halloween, it was removed from the App Store December 7th 2019 due to not having an update for a substantial period of time.
\nAs part of my year of small I wanted to make this available again, and at the same time setup fastlane to automate the screenshots and store the app metadata to make future updates easier.
\npastelghouls was created by my friend Joshua Robins and published by my company Yetii Ltd.
", "url": "https://josephduffy.co.uk/posts/pastelghouls-available-again", "title": "pastelghouls Available Again", "date_modified": "2022-01-16T20:26:14.000Z", "date_published": "2022-01-16T20:26:14.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/year-of-small", "content_html": "I've become accustomed to using yearly themes – rather than New Years resolution – thanks to CGP Grey, who has a very good summary of yearly themes.
\nIt took me a while to home in on it but this year the title of my theme will be "Year of Small."
\nI'll be focusing on all things small:
\nAt the end of the year I'll post a "year in review," although I hope to revisit this every 3 months to make sure I'm still focusing on the right things.
\nAnything related to this theme will be posted under the year-of-small tag.
", "url": "https://josephduffy.co.uk/posts/year-of-small", "title": "The Year of Small", "date_modified": "2022-01-16T19:54:12.000Z", "date_published": "2022-01-16T19:54:12.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/swift-package-collection-signing-using-the-terminal", "content_html": "Swift Packages are JSON files that describe a collection of packages. This post will explain how to sign these packages with a trusted certificate entirely from the terminal. These methods should work on Linux and macOS alike. At the end I describe how to have Swift on Linux implicitly trust these packages.
\nUsing this technique I have published my own package collection.
\nIf you're targeting macOS only and find GUIs more intuitive I recommend following the “Swift Package Collection” blog post from Shaps, which is the post that finally made this “click” for me.
\nThe first choice to make is what kind of key to generate. Packages support certificates using either 256-bit EC or 2048-bit RSA keys. Check with your certificate provider which they support.
\n\nApple's Code Signing certificates only support 2048-bit RSA keys. If you want to use a 256-bit EC key you'll need to use an alternative certificate provider.\n
\nA 2048-bit RSA key can be generated using:
\nopenssl genrsa -out private.pem 2048\n
\nHere's what this command is doing:
\ngenrsa
tells openssl
to generate an RSA key-out private.pem
tells openssl
to output the generated file to private.pem
2048
tells openssl
to generate a key with 2048 bitsopenssl
can also be used to generate that key.\nopenssl ecparam -name prime256v1 -genkey -noout -out private.pem\n
\nHere's what this command is doing:
\necparam
tells openssl
to work with an EC (elliptical curve) key-name prime256v1
tells openssl
to use the curve named prime256v1
(which is 256-bit)-genkey
tells openssl
to generate a key-noout
prevents including the public key in the output (it's not necessary here)-out private.pem
tells openssl
to output the generated file to private.pem
To have a trusted certificate authority to sign our key we need to generate a Certificate Signing Request (CSR):
\nopenssl req -new -key private.pem -out req.csr\n
\nYou must provide a value for at least 1 field.
\nThis command is doing:
\nreq
specifies we are making a request-new
tells openssl
we want to generate a new request-key private.pem
specifies that we want to request that private.pem
be signed-out req.csr
tells openssl
tou output the request to req.csr
The produced req.csr
file can be uploaded to your chosen certificate authority. If you're using Apple you can do this through their developer portal, specifically https://developer.apple.com/account/resources/certificates/add and choose “Swift Package Collection Certificate.”
\nApple will only provide 1 code signing certificate per paid developer account.\n
\nDownload the .cer
file from your certificate provider and keep it safe. Here I'm going to assume you've named it swift_package.cer
.
With your new certificate you can now sign your package:
\n\nThis is using the swift-package-collection-generator package, which you will need to install separately.\n
\nswift run package-collection-sign collection.json signed-collection.json private.pem swift_package.cer\n
\nThis file can now be uploaded to a server and others can add it.
\n\nNeed to quickly test hosting the file? ngrok http "file://$(pwd)"
can be used to serve the files over HTTPS and provide external routing.\n
At this point your collection will be trusted when added via Xcode, but not when downloaded on Linux. To work on Linux the consumer needs to add the root certificate that signed your certificate to their trust store. This is stored in ~/.swiftpm/config/trust-root-certs/
. To test this we can start a Docker container that contains Swift and try adding the package.
docker run --rm -it --entrypoint bash swift\n
\nThis is asking Docker to:
\nrun
a container--rm
: remove it if it exists--it
: Makes the container --interactive
and attaches a --tty
--entrypoint bash
: Enters the container using bash
swift
: Use the latest Swift image (at the time of writing this is 5.5.2)We can try adding the package without making any changes:
\nswift package-collection add https://example.com/swift-package-collection.json\n
\nThis will display the following error:
\nError: The collection's signature is invalid. If you would still like to add it please rerun 'add' with '--skip-signature-check'.\n
\nTo be able to download the certificate you'll need a tool such as wget
:
apt update\napt install wget\n
\nThe target directory also needs to exist:
\nmkdir -p ~/.swiftpm/config/trust-root-certs/\n
\nYou can get the root certificate from your certificate provider. To see the name of the issuer run openssl x509 -noout -inform DER -in swift_package.cer -issuer
.
With my Apple-provided cert this outputs:
\nissuer= /CN=Apple Worldwide Developer Relations Certification Authority/OU=G3/O=Apple Inc./C=US
This certificate can be found on Apple's PKI. Copy the URL and download it in the container:
\nwget https://www.apple.com/certificateauthority/AppleWWDRCAG3.cer -O ~/.swiftpm/config/trust-root-certs/AppleWWDRCAG3.cer\n
\nIf we rerun the command we should now see:
\nAdded "Joseph Duffy's Collection" to your package collections.
🎉 Our package is now signed and can be trusted by by both macOS and Linux users!
\nNow all you need to do is upload the signed-collection.json
file somewhere and link people to it.
Swift Packages are JSON files that describe a collection of packages. This post will explain how to sign these packages with a trusted certificate entirely from the terminal. These methods should work on Linux and macOS alike. At the end I describe how to have Swift on Linux implicitly trust these packages.
\nUsing this technique I have published my own package collection.
\nIf you're targeting macOS only and find GUIs more intuitive I recommend following the “Swift Package Collection” blog post from Shaps, which is the post that finally made this “click” for me.
", "date_modified": "2022-01-11T05:49:14.000Z", "date_published": "2022-01-11T05:49:14.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/hosting-docc-archives", "content_html": "At WWDC21 Apple introduced DocC, a tool for creating archives of Swift documentation that includes the static files required to host a version of the documentation on a website.
\nIn this post I will summarise various methods of serving a DocC archive:
\nAll the examples provided here are hosting the DocC archive for VaporDocC, the Vapor middleware I wrote for hosting DocC archives.
\nAll the examples implement the same basic set of rules:
\n/documentation/
or /tutorials/
to the index.html
file/css/
, /data/
, /downloads/
, /images/
, /img/
, /index/
, /js/
, or /videos/
to their respective foldersfavicon.ico
, favicon.svg
, and theme-settings.json
to their respective files/data/documentation.json
to the file in the data/documentation/
folder that has the name of the module and ends with .json
/
and /documentation
to /documentation/
/tutorials
to /tutorials/
The redirects aren't strictly necessary, although they will likely be useful to a lot of people.
\nWebsite: docc-netlify.josephduffy.co.uk | Repo: github.com/JosephDuffy/DocC-netlify
\nNetlify is one of the easiest to setup and provides the most functionality for the least amount of setup. To get setup add your DocC archive to a git repo, add the below configuration (updating the name of the .docarchive
), publish your repo (public or private), and then set it up on Netlify.
[build]\npublish = "VaporDocC.doccarchive/"\n\n[[redirects]]\nfrom = "/documentation/*"\nstatus = 200\nto = "/index.html"\n\n[[redirects]]\nfrom = "/tutorials/*"\nstatus = 200\nto = "/index.html"\n\n[[redirects]]\nfrom = "/data/documentation.json"\nstatus = 200\nto = "/data/documentation/vapordocc.json"\n\n[[redirects]]\nforce = true\nfrom = "/"\nstatus = 302\nto = "/documentation/"\n\n[[redirects]]\nforce = true\nfrom = "/documentation"\nstatus = 302\nto = "/documentation/"\n\n[[redirects]]\nforce = true\nfrom = "/tutorials"\nstatus = 302\nto = "/tutorials/"\n
\nNetlify will host your DocC archive and allow you to setup your own domain to point to the documentation, including automatically provisioning and TLS certificate, all for free.
\nWebsite: vapordocc.josephduffy.co.uk |\nRepo: github.com/JosephDuffy/VaporDocC-website
\nVaporDocC is middleware I wrote for use with Vapor, a server-side Swift web framework.
\nI personally like this because the only configuration required is pointing it to the DocC archive; it will automatically find the json
file in data/documentation/
when /data/documentation.json
is requested. It's also entirely written in Swift!
A docker container is published at ghcr.io/josephduffy/vapordocc, which can be used to host your own DocC archive with minimal setup:
\ndocker run \\\n -p 8080:8080 \\\n -v /path/to/ModuleName.doccarchive:/docs \\\n ghcr.io/josephduffy/vapordocc\n
\nThere are 2 environment variables that can be used to configure the redirects, both of which are disabled by default. To meet the basic set of rules outlines above you can use:
\ndocker run \\\n -p 8080:8080 \\\n -v /path/to/ModuleName.doccarchive:/docs \\\n -e REDIRECT_ROOT=/documentation/ \\\n -e REDIRECT_MISSING_TRAILING_SLASH=TRUE \\\n ghcr.io/josephduffy/vapordocc\n
\nWebsite: docc-nginx.josephduffy.co.uk | Repo: github.com/JosephDuffy/DocC-nginx
\nUsing the nginx configuration may be a good option if you're adding documentation or tutorials to a website that's already using nginx. For nginx there are a few location blocks that will satisfy the basic set of rules:
\nlocation ~ ^/(documentation|tutorial)/ {\n alias /docs/;\n try_files /index.html =404;\n}\n\nlocation = /data/documentation.json {\n alias /docs/data/;\n # Xcode 13.0 beta 1 generated DocC archives without a file at data/documentation.json.\n # Replace "vapordocc" with the name of the name of your product.\n try_files /documentation.json /documentation/vapordocc.json =404;\n}\n\nlocation ~ ^/(css|data|downloads|images|img|index|js|videos)/ {\n alias /docs/;\n try_files $uri =404;\n}\n\nlocation ~ ^/(favicon.ico|favicon.svg|theme-settings.json)$ {\n alias /docs/;\n try_files $uri =404;\n}\n\nlocation = / {\n return 302 /documentation/;\n}\n\nlocation = /documentation {\n return 302 /documentation/;\n}\n\nlocation = /tutorial {\n return 302 /tutorial/;\n}\n
\nThis configuration assumes that your DocC archive is stored at /docs/
.
Website: docc-apache.josephduffy.co.uk | Repo: github.com/JosephDuffy/DocC-Apache
\nAn Apache config is the example provided in the WWDC talk, but some extra changes are required to have it function correctly:
\nRewriteEngine On\n\nRewriteRule ^(documentation|tutorials)\\/.*$ ModuleName.doccarchive/index.html [L]\n\n# Xcode 13.0 beta 1 generated DocC archives without a file at data/documentation.json.\n# Replace "modulename" with the name of the name of your product.\nRewriteRule /data/documentation.json ModuleName.doccarchive/data/documentation/modulename.json [L]\n\nRewriteRule ^(css|js|data|images|downloads|img|videos)\\/.*$ ModuleName.doccarchive/$0 [L]\n\nRewriteRule ^(favicon\\.ico|favicon\\.svg|theme-settings\\.json)$ ModuleName.doccarchive/$0 [L]\n\nRewriteRule ^$ /documentation/ [L,R=302]\nRewriteRule ^documentation$ /documentation/ [L,R=302]\nRewriteRule ^tutorials$ /tutorials/ [L,R=302]\n
\nThis configuration assumes that your DocC archive is stored at /usr/local/apache2/htdocs/ModuleName.doccarchive
.
The VaporDocC, nginx, and Apache options don't cover things like TLS (HTTPS) or caching headers. These are left as an exercise for the reader, but I would recommend using Netlify or hosting yourself and using Cloudflare if you're unsure what to do.
\nIf you've watched the WWDC21 talk "Host and automate your DocC documentation" you may have run in to some issues when trying to use the provides an example .htaccess
file:
# Enable custom routing.\nRewriteEngine On\n\n\n# Route documentation and tutorial pages.\nRewriteRule ^(documentation|tutorials)\\/.*$ SlothCreator.doccarchive/index.html [L]\n\n\n# Route files within the documentation archive.\nRewriteRule ^(css|js|data|images|downloads|favicon\\.ico|favicon\\.svg|img|theme-settings\\.json|videos)\\/.*$ SlothCreator.doccarchive/$0 [L]\n
\nFor example the favicon.ico
, favicon.svg
, and theme-settings.json
files are not handled because the rewrite rule requires a trailing slash.
Another issue is that the DocC archive generated by Xcode 13.0 beta 1 generates a file at data/documentation/$module_name.json
, but the generated HTML file requests a file at data/documentation.json
. I am hoping a future release generates the file at data/documentation.json
to allow for generic solutions to work without the need to manually provide the module name.
So far I'm loving DocC. The websites produces are really nice, with support for both light and dark mode, and the option of writing documentation within the source files themselves while embedding the linking between symbols within the documentation has made it really pleasant to write documentation.
\nI hope that with it being so easy to write at least some documentation we see more documentation in open source projects, and I hope that this posts helps people with hosting that documentation.
\nIf you've found this useful I'd love for you to tell me about it on Twitter! If you think there could be improvements to any of the solutions please open an issue or a pull requests against the repo containing the example.
", "url": "https://josephduffy.co.uk/posts/hosting-docc-archives", "title": "Hosting DocC Archives", "summary": "At WWDC21 Apple introduced DocC, a tool for creating archives of Swift documentation that includes the static files required to host a version of the documentation on a website.
\nIn this post I will summarise various methods of serving a DocC archive:
\nAll the examples provided here are hosting the DocC archive for VaporDocC, the Vapor middleware I wrote for hosting DocC archives.
", "date_modified": "2021-06-12T21:49:13.000Z", "date_published": "2021-06-12T21:49:13.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/handling-ical-files-in-ios", "content_html": "The iCal format, first defined as a standard as RFC 2445 in 1998, is the universally accepted format for distributing calendar files, mainly used for distributing events.
\nAs part of my QR code scanning app Scanula I added support for detecting events in scanned objects. Thanks to the fantastic libical and the Swift wrapper swift-ical it's fairly easy to parse an iCal feed, but adding it to iOS is a bit trickier.
\nThe best solution I have come up with is to host a small HTTP server within the app that redirects all requests to a data:
URL containing the iCal text. Hosting a server to serve a single file to be able to add calendar events may sound a bit farfetched. To explain why I think this is the best solution I'll first go over the alternatives, with the final solution at the end.
In versions prior to 1.2 of Scanula I would create a new EKEventEditViewController
, passing in a few pieces of information from the scanned iCal text. This is done by creating an EKEvent
object and populating the various properties. This has a few downside though:
EKEvent
So I started looking in to alternatives. The first thing that I tried was to write the iCal text to a temporary file and then providing the URL to a UIDocumentInteractionController
. I thought this would offer the user the option to add the event to their calendar, however, the bottom bar is simply missing. This isn't a commonly reported issue, but there others commenting on this (1, 2). Pre-requesting access to the calendar does not fix this; I assume that Apple's Safari and Mail can do this thanks to a special entitlement.
I thought that my best bet was to handoff the handling to Safari, but since the iCal text has been scanned via an image I can't provide a web URL for Safari to open. Since Shortcuts has special permissions it can open Safari with a non-http(s) URL, e.g. a data:
URL. So I created a Shortcut that opens Safari with a hardcoded data:text/calendar;base64,{base64-encoded-ical}
URL. It worked! But how can I have this be done by my own app?
SFSafariViewController
has the same limitation and will crash when opening a non-http(s) URL. This is where the local web server comes in. By hosting a web server on port 8080 within the app and passing in localhost:8080
as the URL it will query the local web server. All this web server does it redirect to data:text/calendar;base64,{base64-encoded-ical}
and then stop itself. For this I used Embassy because it seemed the most lightweight.
The UX of this isn't quite perfect because it will present an SFSafariViewController
, which will then present the view controller containing the calendar event. When the calendar event view controller is dismissed the user is left with an empty SFSafariViewController
, but I feel that these are all fair tradeoffs.
I do this with a fairly simple server:
\nimport Embassy\nimport Foundation\n\npublic final class RedirectionServer {\n public private(set) static var shared = RedirectionServer()\n\n private var loop: EventLoop?\n private var server: HTTPServer?\n private var loopQueue: DispatchQueue?\n\n public func redirectNextRequestTo(_ url: URL) throws -> URL {\n tearDown()\n\n let loop = try SelectorEventLoop(selector: try KqueueSelector())\n let server = DefaultHTTPServer(eventLoop: loop, port: 8080) { [weak self] _, startResponse, sendBody in\n startResponse("302", [\n ("Location", url.absoluteString),\n ])\n // Empty data ends response\n sendBody(Data())\n\n self?.tearDown()\n }\n\n try server.start()\n\n let loopQueue = DispatchQueue(label: "Redirection Server")\n\n self.loop = loop\n self.server = server\n self.loopQueue = loopQueue\n\n loopQueue.async {\n loop.runForever()\n }\n\n return URL(string: "http://localhost:8080")!\n }\n\n private func tearDown() {\n server?.stop()\n loop?.stop()\n\n server = nil\n loop = nil\n loopQueue = nil\n }\n}\n
\nPresenting the UI to the user is also fairly simple:
\nlet data = Data(iCalString.utf8)\nlet base64Calendar = data.base64EncodedString()\nlet calendarDataURL = URL(string: "data:text/calendar;base64," + base64Calendar)!\n\nDispatchQueue.global().async {\n do {\n let redirectionURL = try RedirectionServer.shared.redirectNextRequestTo(calendarDataURL)\n\n DispatchQueue.main.async {\n let safariViewController = SFSafariViewController(url: redirectionURL)\n safariViewController.modalPresentationStyle = .formSheet\n parentViewController.present(safariViewController, animated: true, completion: nil)\n }\n } catch {\n print("Failed to start redirection server:", error)\n }\n}\n
",
"url": "https://josephduffy.co.uk/posts/handling-ical-files-in-ios",
"title": "Handling iCal Files in iOS",
"summary": "The iCal format, first defined as a standard as RFC 2445 in 1998, is the universally accepted format for distributing calendar files, mainly used for distributing events.
\nAs part of my QR code scanning app Scanula I added support for detecting events in scanned objects. Thanks to the fantastic libical and the Swift wrapper swift-ical it's fairly easy to parse an iCal feed, but adding it to iOS is a bit trickier.
", "date_modified": "2021-06-06T19:50:47.000Z", "date_published": "2021-06-06T19:50:47.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/the-ipod-touch-is-my-favourite-device-for-ios-development", "content_html": "It's important to test across various screen sizes, which the iOS simulator is good for, but it's also important to test on real devices where possible. I currently have an iPhone 11 Pro, an iPhone 6, and 2 iPod touches. Out of all these I find the iPod touch to be the best device for a lot of iOS development.
\nEspecially when starting out developing any extra cost can be a burden. The often-recommended iPhone SE starts at $399/£399 but the iPod touch starts at $199/£199 which makes it a comparatively great deal.
\nFor the same price as an iPhone SE you can have an iPod touch kept on the latest developer beta and another on an older version, giving you a lot more OS version coverage.
\nThe iPod touch has the same screen size as the zoomed iPhone SE, so if the design fits on an iPod touch you know it works for the smallest screen size available. Testing on larger screens is also important but this can be done in the simulator, or your own personal iPhone.
\nThe iPod touch has an A10 processor, while the iPhone SE has an A13 processor. There are iPhones and iPads that still receive iOS updates and have slower processors, although none of these are currently sold by Apple.
\nIt's not perfect but it's going to be easier to find performance issues with an iPod touch than any other iOS or iPadOS devices currently sold by Apple.
\nThe iPod touch comes in 6 different colours, the iPhone SE comes in 3 colours. 6 > 3.
\nThe biggest downside to the iPod touch is a lack of hardware features. It only comes with WiFi, a front camera, a back camera, bluetooth, a three-axis gyro, and an accelerometer. This means it's missing:
\nFor a lot of apps this is unlikely to pose a problem but if your app relies on one of these it may make the iPod touch a bad choice for app development.
\nIdeally I would like to see an iOS device or iOS version specifically aimed at developers that could emulate a lot of these things, e.g. slowing down the processor, disabling hardware, reducing the resolution of the screen, install older iOS versions.
\nApple offer the security research device so it may be a possibility in the future, but for now I'm going to stick buying iPod touches.
", "url": "https://josephduffy.co.uk/posts/the-ipod-touch-is-my-favourite-device-for-ios-development", "title": "The iPod touch Is My Favourite Device for iOS Development", "summary": "It's important to test across various screen sizes, which the iOS simulator is good for, but it's also important to test on real devices where possible. I currently have an iPhone 11 Pro, an iPhone 6, and 2 iPod touches. Out of all these I find the iPod touch to be the best device for a lot of iOS development.
", "date_modified": "2021-05-12T23:37:34.000Z", "date_published": "2021-05-12T23:37:34.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/wwdc-2021-wishlist", "content_html": "With WWDC 2021 just around the corner I've been thinking about what I'd like to see there.
\nA lot of the popular discourse around this time of year is focussed on features of the operating systems but I want to look at what I'd like to see as a developer for Apple platforms.
\nI love to develop for Apple platforms but it can often be a painful process. May is like a christmas for Apple developers.
\nThe App Store Connect API has proved itself very useful by allowing the use of API keys to access to various App Store Connect features, such as uploading and submitting apps and managing certificates.
\nIf it were integrated in to Xcode it could enable downloading signing certificates and provisioning profiles without logging in. This could make deployment from continuous deployment servers easier without the need for external tools like fastlane.
\nThese changes are almost guaranteed since they're mentioned in the Changelog for Swift.
\nI've converted one of my projects to use async
/await
, which surfaced a bug and reduced the number of lines of code. Migrating some of the thread-safe code to use actors will further reduce the complexity and likely improve performance and possible remove more bugs, so I'm very excited for this!
Xcode extensions are currently quite limited, only having access to the currently open file and devoid of invocation methods outside of the menu bar/keyboard shortcuts.
\nExpanding the capabilities to allow triggering invocation on save, adding UI to the editor, and providing syntax highlighting could enable a plethora of new extensions. These could allow Apple to focus on the core IDE experience and debugging tools while leaving the nice-to-haves up to the community, such as:
\nApple could even provide some of these extensions and update them independently of Xcode releases.
\nWith rumours of a portless iPhone on the horizon wireless debugging will need to see some drastic improvements.
\nThe main limitation of wireless debugging seems to be data throughput so I'm guessing that these would be not be sold as wireless debugging improvements but overall improvements to debugging speed, such as by reducing the data that needs to be sent to the device.
\nThe console in Xcode is used for both viewing logs and interfacing with the debugger, but the logs side of this can be quite cumbersome when the only filter option is a single text field.
\nMost apps I work on require using Console.app, which is a decent app, but it would be nice to have these features integrated in to Xcode.
\nSwiftUI is still my favourite changes of the last few years. I was ecstatic when it was announced. But there are a lot of areas that still need improving.
\nSwiftUI was sold as a UI framework with cross platform support, but that quickly falls apart if you want write a SwiftUI-only app.
\nMy biggest gripe occurs when trying to use a modifier that is only supported on a subset of platforms, such as tap gestures and navigation title styles. Making these no-ops on unsupported platforms (similar to what Catalyst does on macOS) or providing a method to only apply a modifier on a specific platform would be very welcomed.
\nThe Swift evolution pitch "‘#if’ for postfix member expressions" could fix this but it's not yet been reviewed.
\nSome SwiftUI views do not have feature parity with their UIKit or AppKit counterparts. For example setting the preferred display mode for a split view is not possible. SwiftUI tries to be clever and use a UISplitViewController
when the content of a NavigationView
contains more than 1 view, but it's not possible to then control how these are shown.
A lot of the time saved by using SwiftUI can be lost to silly things like this requiring a fallback to UIKit or AppKit, which can lead to a different set of bugs and limitations. Hopefully some of these gaps are filled this year!
\nA large chunk of the download size of Xcode is the support for various platforms.
\nThey appear to be compatible with each other and symlinking the runtimes is recommended when using GitHub Actions, so in theory they could be store centrally.
\nThis could reduce the initial download size of Xcode and ease updating. Sometimes I only care about a Swift update or a new IDE feature and I don't need the latest tvOS runtime, so why do I have to download it?
\nSince Apple bought buddybuild in early 2018 I've been hoping it would go a similar way to the TestFlight acquisition and be integrated in to their developer services. It's been 3 years and I'm still hoping for this, but I'm putting this in the moonshot category because I now think if Apple were to provide this they'd want the servers to run on Apple Silicon and the current hardware offerings aren't aimed at the server market, although the M1 Mac Mini could be a candidate.
\nI love generics a little too much and often abuse them. While I aim to improve my use of generics – often by removing them – unlocking existential types would be a huge win.
\nThere's an evolution pitch (Lifting the “Self or associated type” constraint on existentials) that's been discussing this for a few years but I'm still hopeful it could make its way in to Swift 6.
\nApple's quality and quantity of documentation has fallen in recent years and while we've seen some improvements lately I have a feeling WWDC 2021 isn't going to fix all of this.
\nI don't believe Apple would've been holding back on us when it comes to documentation updates, especially since there have been updates recently, such as adding missing documentation to some Combine APIs.
\nI would love to be wrong about this and see the majority of new APIs have clear documentation when the Xcode 13 beta is available but I'm still classing this as a moonshot.
\nCurrently the vast majority of bug fixes are fixed by OS updates. Although the adoption rate of iOS updates is high it's not something that can be relied on.
\nA recent example of this the bug fix that now allows multiple sheet(isPresented:onDismiss:content:)
and fullScreenCover(item:onDismiss:content:)
modifiers in the same view hierarchy, as long as the user is running iOS 14.5 beta 3 or later. This means that without bumping the deployment target the fix cannot be use. This is a bug that's very easy to miss in testing, especially if you're unaware of it.
For some APIs this could be a shim layer that gets downloaded with the first app that uses the API and then dynamically linked at app launch. I'm not familiar enough with how this could work on a technical level to say if this is viable but I know it would be massively loved by developers!
\nThe Apple Security Research Device Program allows security researchers to own an iPhone that provides shell access. This level of access would not be required for developers. I would be very happy with support for downgrading the OS version and more on-device options for debugging such as simulating the location of device without Xcode.
\nWhile working on macOS apps I find the defaults
command very useful. Having something similar on iOS would surely prove itself useful too. This may require the sandbox to be loosened and so feels unlikely, although if this were allowed there may be many more developer-focussed apps that it would enable.
The Tree Style Tab Firefox extension makes navigating more than a few tabs manageable. I find this most useful when researching something, such as when looking at a new API and utilising different blog posts, documentation, etc.
\nThis a real moonshot because I don't see Apple adding this as a native feature to Safari, nor do I see them opening the extensions API to allow a 3rd party to develop this.
\nFor now I'll likely be sticking with Firefox!
\nWriting this post has made me very excited for WWDC 2021, which is looking to be the second virtual conference in a row. Personally I like the virtual format but I'm looking forward to the possibility of attending in person again!
", "url": "https://josephduffy.co.uk/posts/wwdc-2021-wishlist", "title": "My WWDC 2021 Wishlist", "summary": "With WWDC 2021 just around the corner I've been thinking about what I'd like to see there.
\nA lot of the popular discourse around this time of year is focussed on features of the operating systems but I want to look at what I'd like to see as a developer for Apple platforms.
\nI love to develop for Apple platforms but it can often be a painful process. May is like a christmas for Apple developers.
", "date_modified": "2021-03-19T14:53:22.000Z", "date_published": "2021-03-04T12:52:34.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/mapping-optional-binding-to-bool", "content_html": "When displaying an alert in SwiftUI, if the value used to calculate whether the alert is presented is both Optional
and does not conform to Identifiable
1 it is often recommended to use a separate flag, similar to:
struct ContentView: View {\n @State private var alertText: String?\n @State private var isPresentingAlert = false\n\n var body: some View {\n Button("Show Alert") {\n self.alertText = "Alert Text"\n self.isPresentingAlert = true\n }\n .alert(isPresented: $isPresentingAlert) {\n Alert(title: Text(alertText!))\n }\n }\n}\n
\nThere are 2 main downsides to this:
\nalertText
is not set back to nil
, which may cause bugs and will increase memory usage (even if only a little in this case)isPresentingAlert
flag needs to be managedTo work around these issues I create a small extension to Binding
the allows this same code to be updated to:
struct ContentView: View {\n @State private var alertText: String?\n\n var body: some View {\n Button("Show Alert") {\n self.alertText = "Alert Text"\n }\n .alert(isPresented: $alertText.mappedToBool()) {\n Alert(title: Text(alertText!))\n }\n }\n}\n
\nThe extension is fairly small and simple:
\nimport os.log\nimport SwiftUI\n\nextension Binding where Value == Bool {\n /// Creates a binding by mapping an optional value to a `Bool` that is\n /// `true` when the value is non-`nil` and `false` when the value is `nil`.\n ///\n /// When the value of the produced binding is set to `false` the value\n /// of `bindingToOptional`'s `wrappedValue` is set to `nil`.\n ///\n /// Setting the value of the produce binding to `true` does nothing and\n /// will log an error.\n ///\n /// - parameter bindingToOptional: A `Binding` to an optional value, used to calculate the `wrappedValue`.\n public init<Wrapped>(mappedTo bindingToOptional: Binding<Wrapped?>) {\n self.init(\n get: { bindingToOptional.wrappedValue != nil },\n set: { newValue in\n if !newValue {\n bindingToOptional.wrappedValue = nil\n } else {\n os_log(\n .error,\n "Optional binding mapped to optional has been set to `true`, which will have no effect. Current value: %@",\n String(describing: bindingToOptional.wrappedValue)\n )\n }\n }\n )\n }\n}\n\nextension Binding {\n /// Returns a binding by mapping this binding's value to a `Bool` that is\n /// `true` when the value is non-`nil` and `false` when the value is `nil`.\n ///\n /// When the value of the produced binding is set to `false` this binding's value\n /// is set to `nil`.\n public func mappedToBool<Wrapped>() -> Binding<Bool> where Value == Wrapped? {\n return Binding<Bool>(mappedTo: self)\n }\n}\n
\nThe extension isn't tied directly to showing an alert or a sheet and can be used in any context, but this is one of the better examples of its usage.
\nThis extension is available on GitHub under the MIT license.
\n1 If it does conform to Identifiable
use alert(item:content:)
The Xcode 12 beta includes Swift 5.3 but drops support for iOS 8.x. This means that Swift packages that support iOS 8 will cause a warning:
\n\n\nThe iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99.
\n
It's not possible to remove this warning within a project that depends on a Swift package with a deployment target of iOS 8, but it is possible to fix this in the dependency without removing support for iOS 8 for older versions of Swift. There are multiple way this can be accomplished.
\nI was not able to find documentation on version-specific manifests, but the source code shows they take priority over the default Package.swift
file and will search for files in the following order:
In this example a new Package@swift-5.3.swift
file can be added with a small change:
-// swift-tools-version:5.2\n+// swift-tools-version:5.3\n
\n platforms: [\n- .iOS(.v8),\n+ .iOS(.v9),\n ],\n
\nThis change is arguably not a breaking change so does not require a new major version of the package but it still fixes the warning. It could be argued that someone may use Xcode 12 with an older version of Swift or an older version of Xcode with a newer version of Swift, but the App Store would not accept an app built a Swift toolchain that did not come with the download of Xcode.
\nI have made this change in a personal project (Persist) without any issues, and have also opened PRs on 2 projects that I use: DeviceKit and KeychainAccess.
\n#if compiler
Compilation ConditionA second option is to use #if compiler
to update the supported platforms. This has the advantage of not requiring multiple package files, but can also lead to confusing if the package file is large and the condition is easy to miss.
This can be added after the let package =
declaration:
#if compiler(>= 5.3)\npackage.platforms = [.iOS(.v9)] // Ensure to add any other platforms you support here too!\n#endif\n
\nNote that #if compiler
is used rather than #if swift
to allow for newer compilers that are compiling code using an older Swift language version. This change was pointed out to me when I was making this change as part of a PR on the CombineX project to fix the warning in Xcode 12.
The Xcode 12 beta includes Swift 5.3 but drops support for iOS 8.x. This means that Swift packages that support iOS 8 will cause a warning:
\n\n\nThe iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99.
\n
It's not possible to remove this warning within a project that depends on a Swift package with a deployment target of iOS 8, but it is possible to fix this in the dependency without removing support for iOS 8 for older versions of Swift. There are multiple way this can be accomplished.
", "date_modified": "2020-10-25T16:53:42.000Z", "date_published": "2020-08-11T19:08:20.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/running-ui-tests-on-mac-catalyst", "content_html": "While working on the 2.0 update for Gathered I have been trying to develop the app multiple platforms simultaneously. SwiftUI will solve this problem in the future, but I wish to support some OS versions that SwiftUI does not support.
\nAs part of this I have been creating UI tests to test performance, but ran in to an issue when running the UI tests on macOS using Mac Catalyst:
\nRunning tests...\nThe bundle “PerformanceXCTests” couldn’t be loaded because it is damaged or missing necessary resources. Try reinstalling the bundle.\n(dlopen_preflight(...): no suitable image found. Did find:\n\t...: code signature in (...) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?))\n
\nThe solution to this problem (as hinted in the error) is to opt out of library validation. This can be done using the Disable Library Validation entitlement, but when editing the test target in Xcode the Signing & Capabilities tab doesn't have the Hardened Runtime section.
\nTo get around this a custom entitlements file can be created:
\n<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">\n<plist version="1.0">\n<dict>\n\t<key>com.apple.security.cs.disable-library-validation</key>\n\t<true/>\n</dict>\n</plist>\n
\nThis file can saved anywhere (I have it at the root of the tests directory) and then set as the value for the Code Signing Entitlements (CODE_SIGN_ENTITLEMENTS
) option in the Build Settings tab.
Re-run the tests with My Mac selected and your UI tests should run using Mac Catalyst.
", "url": "https://josephduffy.co.uk/posts/running-ui-tests-on-mac-catalyst", "title": "Running UI Tests on Mac Catalyst", "summary": "While working on the 2.0 update for Gathered I have been trying to develop the app multiple platforms simultaneously. SwiftUI will solve this problem in the future, but I wish to support some OS versions that SwiftUI does not support.
\nAs part of this I have been creating UI tests to test performance, but ran in to an issue when running the UI tests on macOS using Mac Catalyst:
\nRunning tests...\nThe bundle “PerformanceXCTests” couldn’t be loaded because it is damaged or missing necessary resources. Try reinstalling the bundle.\n(dlopen_preflight(...): no suitable image found. Did find:\n\t...: code signature in (...) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?))\n
",
"date_modified": "2020-07-11T17:51:00.000Z",
"date_published": "2020-07-11T17:51:00.000Z",
"author": {
"name": "Joseph Duffy",
"url": "https://josephduffy.co.uk/"
}
},
{
"id": "https://josephduffy.co.uk/posts/my-swiftpm-release-workflow",
"content_html": "I am currently maintaining numerous Swift Packages that don't receive a constant flow of updates, but do receive updates when new Swift updates come out, or as I think of useful additions.
\nTo ensure that I can make some of these less frequent updates without too much friction and with confidence in their correctness I rely heavily on GitHub Actions, which I'll go over in this blog post.
\nI have 2 workflows that I use across my projects, one for running tests and the other for performing releases.
\nThe tests workflow runs on every commit.
\nname: Tests\n\non: [push]\n\njobs:\n macos_tests:\n name: macOS Tests (SwiftPM)\n runs-on: macos-latest\n strategy:\n fail-fast: false\n matrix:\n xcode: ["11.4"]\n\n steps:\n - uses: actions/checkout@v2\n\n - name: Select Xcode ${{ matrix.xcode }}\n run: sudo xcode-select --switch /Applications/Xcode_${{ matrix.xcode }}.app\n\n - name: Cache SwiftPM\n uses: actions/cache@v1\n with:\n path: .build\n key: ${{ runner.os }}-xcode_${{ matrix.xcode }}-swiftpm-deps-${{ github.workspace }}-${{ hashFiles('Package.resolved') }}\n restore-keys: |\n ${{ runner.os }}-xcode_${{ matrix.xcode }}-swiftpm-deps-${{ github.workspace }}\n\n - name: SwiftPM tests\n run: swift test --enable-code-coverage\n\n - name: Convert coverage to lcov\n run: xcrun llvm-cov export -format="lcov" .build/debug/PersistPackageTests.xctest/Contents/MacOS/PersistPackageTests -instr-profile .build/debug/codecov/default.profdata > coverage.lcov\n\n - name: Upload coverage to Codecov\n uses: codecov/codecov-action@v1\n with:\n fail_ci_if_error: true\n\n xcode_tests:\n name: ${{ matrix.platform }} Tests (Xcode)\n runs-on: macos-latest\n strategy:\n fail-fast: false\n matrix:\n xcode: ["11.4"]\n platform: ["iOS", "tvOS"]\n\n steps:\n - uses: actions/checkout@v2\n\n - name: Select Xcode ${{ matrix.xcode }}\n run: sudo xcode-select --switch /Applications/Xcode_${{ matrix.xcode }}.app\n\n - name: Cache SwiftPM\n uses: actions/cache@v1\n with:\n path: CIDependencies/.build\n key: ${{ runner.os }}-xcode_${{ matrix.xcode }}-swiftpm-ci-deps-${{ github.workspace }}-${{ hashFiles('CIDependencies/Package.resolved') }}\n restore-keys: |\n ${{ runner.os }}-xcode_${{ matrix.xcode }}-swiftpm-ci-deps-${{ github.workspace }}\n\n - name: Cache DerivedData\n uses: actions/cache@v1\n with:\n path: ~/Library/Developer/Xcode/DerivedData\n key: ${{ runner.os }}-${{ matrix.platform }}_derived_data-xcode_${{ matrix.xcode }}\n restore-keys: |\n ${{ runner.os }}-${{ matrix.platform }}_derived_data\n\n - name: Run Tests\n run: swift run --configuration release --skip-update --package-path ./CIDependencies/ xcutils test ${{ matrix.platform }} --scheme Persist --enable-code-coverage\n\n - name: Upload coverage to Codecov\n uses: codecov/codecov-action@v1\n with:\n fail_ci_if_error: true\n\n watchos_build:\n name: watchOS Build (Xcode)\n runs-on: macos-latest\n strategy:\n fail-fast: false\n matrix:\n xcode: ["11.4"]\n\n steps:\n - uses: actions/checkout@v2\n\n - name: Select Xcode ${{ matrix.xcode }}\n run: sudo xcode-select --switch /Applications/Xcode_${{ matrix.xcode }}.app\n\n - name: Cache SwiftPM\n uses: actions/cache@v1\n with:\n path: CIDependencies/.build\n key: ${{ runner.os }}-xcode_${{ matrix.xcode }}-swiftpm-ci-deps-${{ github.workspace }}-${{ hashFiles('CIDependencies/Package.resolved') }}\n restore-keys: |\n ${{ runner.os }}-xcode_${{ matrix.xcode }}-swiftpm-ci-deps-${{ github.workspace }}\n\n - name: Cache DerivedData\n uses: actions/cache@v1\n with:\n path: ~/Library/Developer/Xcode/DerivedData\n key: ${{ runner.os }}-watchOS_derived_data-xcode_${{ matrix.xcode }}\n restore-keys: |\n ${{ runner.os }}-watchOS_derived_data\n\n - name: Build for watchOS\n run: swift run --configuration release --skip-update --package-path ./CIDependencies/ xcutils build watchOS --scheme Persist\n\n linux_tests:\n name: SwiftPM on ${{ matrix.os }}\n runs-on: ${{ matrix.os }}\n strategy:\n fail-fast: false\n matrix:\n os: [ubuntu-16.04, ubuntu-latest]\n swift: ["5.2.3"]\n\n steps:\n - uses: actions/checkout@v2\n\n - name: Install swiftenv\n run: |\n eval "$(curl -sL https://swiftenv.fuller.li/install.sh)"\n echo "::set-env name=SWIFTENV_ROOT::$HOME/.swiftenv"\n echo "::add-path::$SWIFTENV_ROOT/bin:$PATH"\n\n - name: swift test\n run: swift test --enable-test-discovery\n
\nThe tests are split in to 4 sections:
\nswift test
swift test
on UbuntumacOS, iOS, and tvOS tests gather test coverage and upload it to Codecov, which provides some insight it to how much new code is covered by tests.
\nFor the iOS and tvOS tests, along with the watchOS build, I used xcutils
. xcutils
is another tool of mine that is used to improve the CLI of Xcode. Here it is used to run the tests/build against the latest versions of iOS/tvOS/watchOS, which means it should work on any machine and is resistant to changes made by GitHub.
On Linux the --enable-test-discovery
flag is passed to swift test
to remove the need for a LinuxMain.swift
file that much be kept in sync with the tests.
These tests have helped me match many mistakes before merging, especially for platforms such as watchOS and Linux that are less frequently used.
\nThe release workflow is triggered by the creation of a git tag that starts with a v
.
name: Release\n\non:\n push:\n tags:\n - "v*"\n\njobs:\n create_release:\n name: Create Release\n runs-on: ubuntu-latest\n\n steps:\n - uses: actions/checkout@v2\n\n - name: Fetch tag\n run: git fetch --depth=1 origin +${{ github.ref }}:${{ github.ref }}\n\n - name: Get the release version\n id: release_version\n run: echo "::set-output name=version::${GITHUB_REF/refs\\/tags\\//}"\n\n - name: Get release description\n run: |\n description="$(git tag -ln --format=$'%(contents:subject)\\n\\n%(contents:body)' ${{ steps.release_version.outputs.version }})"\n # Fix set-output for multiline strings: https://github.community/t/set-output-truncates-multiline-strings/16852\n description="${description//'%'/'%25'}"\n description="${description//$'\\n'/'%0A'}"\n description="${description//$'\\r'/'%0D'}"\n echo "$description"\n echo "::set-output name=description::$description"\n id: release_description\n\n - name: Create Release\n id: create_release\n uses: actions/create-release@v1\n env:\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n with:\n tag_name: ${{ steps.release_version.outputs.version }}\n release_name: ${{ steps.release_version.outputs.version }}\n body: ${{ steps.release_description.outputs.description }}\n prerelease: ${{ startsWith(steps.release_version.outputs.version, 'v0.') || contains(steps.release_version.outputs.version, '-') }}\n\n build_docs:\n name: Build Docs\n runs-on: macos-latest\n strategy:\n fail-fast: false\n matrix:\n xcode: ["11.4"]\n\n steps:\n - uses: actions/checkout@v2\n\n - name: Select Xcode ${{ matrix.xcode }}\n run: sudo xcode-select --switch /Applications/Xcode_${{ matrix.xcode }}.app\n\n - name: Setup Ruby\n uses: ruby/setup-ruby@v1\n\n - uses: actions/cache@v1\n with:\n path: vendor/bundle\n key: ${{ runner.os }}-gems-${{ hashFiles('.ruby-version') }}-${{ hashFiles('**/Gemfile.lock') }}\n restore-keys: |\n ${{ runner.os }}-gems-${{ hashFiles('.ruby-version') }}-\n\n - name: Bundle install\n run: |\n bundle config path vendor/bundle\n bundle install --jobs 4 --retry 3\n\n - name: Build docs\n run: bundle exec jazzy\n\n - name: Upload Docs\n uses: peaceiris/actions-gh-pages@v3\n with:\n github_token: ${{ secrets.GITHUB_TOKEN }}\n publish_dir: docs\n
\nThe first job creates a GitHub release for the tag. The release uses the contents of the tag as the body for the release, which allows for markdown, so I write the body of the tag using markdown to benefit from improved rendering on the GitHub website. The Get release description
step modifies the body by escaping new lines and %
characters. This is required to prevent the output being truncated. See https://github.community/t/set-output-truncates-multiline-strings/16852.
Since my releases follow sematic versioning 2.0.0 if the release starts with v0.
or contains a -
the release is marked as pre-release.
The second job runs jazzy
to build HTML docs and uploads it to a gh-pages
branch, which is configured to be deployed automatically by GitHub, and also provides a badge displaying the percentage of public code that is documented.
With these workflows in place I can make a change, add some tests, push, create a pull request, merge, and tag a new release with confidence and all within a couple of hours.
\nSince this workflow will be receiving small tweaks over time and I may not remember to update this workflow straight away (maybe I should make a workflow for that 🤪) you should check out the Persist workflows to find my latest changes.
", "url": "https://josephduffy.co.uk/posts/my-swiftpm-release-workflow", "title": "My Swift Package Manager Release Workflow", "summary": "I am currently maintaining numerous Swift Packages that don't receive a constant flow of updates, but do receive updates when new Swift updates come out, or as I think of useful additions.
\nTo ensure that I can make some of these less frequent updates without too much friction and with confidence in their correctness I rely heavily on GitHub Actions, which I'll go over in this blog post.
", "date_modified": "2020-06-18T21:53:03.000Z", "date_published": "2020-06-18T21:53:03.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/capturing-more-than-self", "content_html": "A common pattern when using closures in Swift is to add [weak self]
in the captures list to hold a weak reference to self
and avoid a retain cycle. This is then often followed by the following:
guard let self = self else { return }\n
\nBut I often forget that capture lists can capture other variables in the current scope, so I thought I'd highlight some other use cases.
\nWhen creating a closure in mutating function of a struct
capturing self
is not possible:
struct Foo {\n var bar: Bool\n\n mutating func createClosure() -> () -> Bool {\n return { // Error: Escaping closure captures mutating 'self' parameter\n return self.bar\n }\n }\n}\n\nvar foo = Foo(bar: true)\nlet closure = foo.createClosure()\nclosure()\n
\nTo work around this you can capture the property:
\nstruct Foo {\n var bar: Bool\n\n mutating func createClosure() -> () -> Bool {\n return { [bar] in\n return bar\n }\n }\n}\n\nvar foo = Foo(bar: true)\nlet closure = foo.createClosure()\nclosure() // true\n
\nAnother use case is capturing a variable a time of closure creation, when it can also be useful to rename the variable:
\nclass Foo {\n var bar: Bool\n\n init(bar: Bool) { self.bar = bar }\n\n func doSomething() {\n let closure = { [weak self, originalBar = bar] in\n guard let self = self else { return }\n\n if originalBar != self.bar {\n print("Bar has changed")\n } else {\n print("Bar did not change")\n }\n }\n\n bar.toggle()\n\n closure()\n }\n}\n\nlet foo = Foo(bar: true)\nfoo.doSomething()\n
\nThe above code will print Bar has changed
.
To read more in-depth information about closures read the Swift language guide page on closures.
", "url": "https://josephduffy.co.uk/posts/capturing-more-than-self", "title": "Capturing More Than `self`", "summary": "A common pattern when using closures in Swift is to add [weak self]
in the captures list to hold a weak reference to self
and avoid a retain cycle. This is then often followed by the following:
guard let self = self else { return }\n
\nBut I often forget that capture lists can capture other variables in the current scope, so I thought I'd highlight some other use cases.
", "date_modified": "2020-05-14T17:58:13.000Z", "date_published": "2020-05-14T17:58:13.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/HashableByKeyPath-framework-release-1-0-0", "content_html": "Today I have released the 1.0.0 version of a Swift package that aids with adding Equatable
and Hashable
conformance by using KeyPath
s.
The package is available on GitHub.
\nI created the Swift Playground that sparked this concept in December 2018, so this concept has been rattling around in my brain for a couple of years. The API has changed a lot since the original concept, but the core has stayed the same: a protocol that requires a single function to be implemented that uses KeyPath
s to synthesise Equatable
and/or Hashable
conformance.
One of the mistakes I've made more times than I care to admit is comparing the wrong properties or objects when adding Equatable
conformance, e.g.:
struct Foo: Equatable {\n static func == (lhs: Foo, rhs: Foo) -> Bool {\n return lhs.bar1 == rhs.bar1 && lhs.bar2 == rhs.bar1 && rhs.bar3 == rhs.bar3\n }\n\n var bar1: String\n var bar2: String\n var bar3: Int\n}\n
\nThese can be easy to type and hard to spot (even with better layout). The EquatableByKeyPath
protocol fixes this issue:
struct Foo: EquatableByKeyPath {\n static func addEquatableKeyPaths<Consumer: EquatableKeyPathConsumer>(to consumer: inout Consumer) where Consumer.Root == Self {\n consumer.addEquatableKeyPath(\\.bar1)\n consumer.addEquatableKeyPath(\\.bar2)\n consumer.addEquatableKeyPath(\\.bar3)\n }\n\n var bar1: String\n var bar2: String\n var bar3: Int\n}\n
\nAdding the addEquatableKeyPaths(to:)
function will synthesise Equatable
conformance. Since you provide every KeyPath
once it's not possible to compare the wrong properties or objects.
Another mistake I've made a lot is not keeping my Equatable
and Hashable
conformance in sync. e.g. I add a new property but only add it to ==
and not hash(into:)
. According to Hashable
, this is bad:
\n\nHashing a value means feeding its essential components into a hash function, represented by the Hasher type. Essential components are those that contribute to the type’s implementation of Equatable. Two instances that are equal must feed the same values to Hasher in hash(into:), in the same order.
\n
Source: https://developer.apple.com/documentation/swift/hashable
\nThe HashableByKeyPath
protocol fixes this issue:
struct Foo: HashableByKeyPath {\n static func addHashableKeyPaths<Consumer: HashableKeyPathConsumer>(to consumer: inout Consumer) where Consumer.Root == Self {\n consumer.addHashableKeyPath(\\.bar1)\n consumer.addHashableKeyPath(\\.bar2)\n consumer.addHashableKeyPath(\\.bar3)\n }\n\n var bar1: String\n var bar2: String\n var bar3: Int\n}\n
\nAdding the addHashableKeyPaths(to:)
function will synthesise Hashable
, EquatableByKeyPath
, and Equatable
conformance. Since you provide every KeyPath
once and only have 1 functions it's not possible to compare the wrong properties or objects or have your Equatable
and Hashable
conformances out of sync.
I will admit that this can't replace every Equatable
and Hashable
conformance out there, but I've not had a scenario yet I had to manually implement ==
or hash(into:)
.
Today I have released the 1.0.0 version of a Swift package that aids with adding Equatable
and Hashable
conformance by using KeyPath
s.
The package is available on GitHub.
\nI created the Swift Playground that sparked this concept in December 2018, so this concept has been rattling around in my brain for a couple of years. The API has changed a lot since the original concept, but the core has stayed the same: a protocol that requires a single function to be implemented that uses KeyPath
s to synthesise Equatable
and/or Hashable
conformance.
Today marks 1 year since I released a blog post demonstrating an implementation of Partial in Swift, and it also marks the release of the 1.0.0 version of a Swift package for Partial.
\nThe package is available on GitHub and supports SwiftPM, Carthage, and CocoaPods.
\nThis blog post will go over some of the changes that have been made since the original blog post, my rationale when making certain decisions, and how I have thought about maintenance. If you want to try out Partial and see how it can be used head over to the GitHub page.
\nSince the original blog post on Partial I have used Partial in production, learned a lot more about Swift, and Swift has received a few updates. Throughout this time I have found some shortcomings and have had more time to think about how some aspects should work.
\nFrom a consumer's point of view one of the main things that have been updated is that there are no longer any possibilities for ambiguity, which were a real problem with the previous implementation. The PartialBuilder
has also been upgraded to provide subscriptions.
Internally the implementation has been simplified, making testing much easier.
\nAs of writing Swift 5.1 is in beta. As part of the 1.0.0 release I have added support for dynamic member lookup when compiling with Swift 5.1, which is a very nice improvement over the subscripting API:
\n// Swift 5.0\npartial[\\.foo]\n\n// Swift 5.1\npartial.foo\n
\nVersion 1.0.0 still supports subscripts, but they are deprecated when using Swift 5.1.
\nPartialBuilder
In the original blog post I mentioned PartialBuilder
, but it was not included in the linked gist. As part of the package release I have included PartialBuilder
. The new implementation include subscriptions to all changes and key path changes.
While at first embedded partials seemed useful I have found that there are better ways to handle setting a partial value on another partial, and it also caused some frustrating ambiguity with the Swift compiler.
\nRather than setting a partial value I found it more useful to attempt to set a partial value. For example, assuming CGSize
conforms to PartialConvertible
, setting a key path of type CGSize
to a partial can be done via setValue(_:for:)
. If the value fails to be unwrapped this function will rethrow the error.
struct SizeWrapper {\n let size: CGSize\n}\nvar wrapperPartial = Partial<SizeWrapper>()\nvar sizePartial = Partial<CGSize>()\ntry wrapperPartial.setValue(sizePartial, for: \\.size) // Will throw an error, value will not be set\nsizePartial.width = 6016\nsizePartial.height = 3384\ntry wrapperPartial.setValue(sizePartial, for: \\.size) // Will set `size` to `CGSize(width: 6016, height: 3384)`\n
\nThis has a few advantages:
\nPartialConvertible
type the implementer does not need to check for partial valuesPartialBuilder
and only be notified when a valid value has been setPartial
does need to check for Partial
valuessetValue(_:for:)
and 2 partialValue(for:)
functions could be removedI do still find the concept of an embedded partial useful, but propose an alternative approach. For example, you may build an instance across multiple screens, setting the values on a single builder by utilising multiple builders.
\nstruct SizeWrapper {\n let size: CGSize\n}\nlet wrapperBuilder = PartialBuilder<SizeWrapper>()\nlet sizeSubscription = wrapperBuilder.subscribeForChanges(to: \\.size) { update in\n print("Size has been set to \\(update.newValue)")\n}\n\nlet sizeBuilder = PartialBuilder<CGSize>()\nlet sizeSubscription = wrapperBuilder.subscribeToAllChanges { _, builder in\n do {\n try wrapperBuilder.setValue(builder, for: \\.size)\n } catch {\n // Optionally remove the value here, or show the error to the user\n wrapperBuilder.removeValue(for: \\.size)\n print("Error unwrapping partial size:", error)\n }\n}\nsizeBuilder.width = 6016\nwrapperBuilder.size // `nil`\nsizeBuilder.height = 3384\nwrapperBuilder.size // `CGSize(width: 6016, height: 3384)`\n
\nDefault values can be implemented by providing a custom unwrapping closure:
\nwrapperBuilder.subscribeToAllChanges { _, builder in\n wrapperBuilder.setValue(builder, for: \\.size) { sizePartial in\n let width = sizePartial.width ?? 6016\n let height = sizePartial.height ?? 3384\n return CGSize(width: width, height: height)\n }\n}\n
\nNote that because the unwrapping closure does not throw the call to setValue(_:for:unwrapper:)
does not need to include try
.
Optional
-specific functionsWhen I created the original version of Partial I found it necessary to create separate functions for handling Optional
s. I am not sure if a Swift update has made the handling of Optional
values better or my understanding around their handling was incorrect, but once I added the test suite and then removed the Optional
-specific functions I found them to be unnecessary.
When I was writing the original blog post autocomplete for key paths was not provided within Xcode. As of writing this blog post this now works in most situations. For example, typing partial.
, partial[\\.]
, or partial.value(for: \\.)
will provide autocomplete, but partial.setValue("new value", for: \\.)
will not. This is a major improvement, but it's still not perfect.
I wanted to get an MVP version 1.0.0 released as soon as possible. Partly to get the project setup complete so I can utilise the structure in other projects, but also because I have a habit of getting carried away with adding new features and never releasing anything.
\nOne of the features I chose not to include in 1.0.0 is support for Combine. This was partly due to the desire to get a 1.0.0 released, but also because iOS 13 (which is required for Combine) is still in beta. I have used Combine in GatheredKit and can see how simple it would be to implement so it is definitely slated for 1.1.0.
\nIn the past I have found that if I take a break from project a I will often to met with a certain amount of maintenance that needs to be performed before I can start work on the changes I want to make. To combat this I have been making some changes to how I setup my projects to make general maintenance easier:
\nHaving full test coverage is a no-brainer; I can feel much more confident about changes that are made. This is especially important for smaller projects like Partial because I will likely take larger breaks between chunks of work.
\nAutomatic deployment is setup via git tags; when a tag is pushed Travis CI will create a pre-built binary for Carthage and attach it to a GitHub release. To remove the development dependencies from the SwiftPM package I have setup Rocket, which makes creating a release as easy as running swift run rocket v1.0.0
. This vastly reduces the friction required for creating releases, meaning updates won't be sitting unreleased for extended periods of time.
Automatic dependency updates have proved worthwhile for me on other projects, so I have added Dependabot to the project. For my TypeScript projects this is much more useful because all dependencies can be updated, but unfortunately Dependabot does not support any Swift dependency managers (neither do any other dependency managers; I am not throwing shade at Dependabot here). This means that only some dependencies, such as those provided by RubyGems, are automatically updated. Nevertheless this is better than nothing.
\nPartial 1.0.0 is a great update compared to the original blog post. It is much easier for me to maintain and is usable in production. If you find any issues, have any questions, and would like to request a feature please open an issue on GitHub or message me on Twitter.
", "url": "https://josephduffy.co.uk/posts/partial-framework-release-1-0-0", "title": "Partial framework release 1.0.0", "summary": "Today marks 1 year since I released a blog post demonstrating an implementation of Partial in Swift, and it also marks the release of the 1.0.0 version of a Swift package for Partial.
\nThe package is available on GitHub and supports SwiftPM, Carthage, and CocoaPods.
\nThis blog post will go over some of the changes that have been made since the original blog post, my rationale when making certain decisions, and how I have thought about maintenance. If you want to try out Partial and see how it can be used head over to the GitHub page.
", "date_modified": "2019-07-10T21:58:41.000Z", "date_published": "2019-07-10T21:58:41.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/gathered-1-3-release-notes", "content_html": "Gathered 1.3 has been released and is now available on the App Store. Version 1.3 brings 2 new data sources, app-wide speed and UX improvements, and support for various features added in recent versions of iOS.
\nThis update also has lots of behind-the-scenes changes that will make future updates easier to create and deploy, which – along with my features roadmap – should mean more frequent updates.
\nI wasn't very happy removing the Heart Rate data source but Apple weren't very happy with the use of HealthKit.
\nGathered 1.3 has been released and is now available on the App Store. Version 1.3 brings 2 new data sources, app-wide speed and UX improvements, and support for various features added in recent versions of iOS.
\nThis update also has lots of behind-the-scenes changes that will make future updates easier to create and deploy, which – along with my features roadmap – should mean more frequent updates.
\nI wasn't very happy removing the Heart Rate data source but Apple weren't very happy with the use of HealthKit.
", "date_modified": "2018-03-11T19:57:27.000Z", "date_published": "2018-03-11T19:57:27.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/ios-share-sheets-the-proper-way-locations", "content_html": "Sharing a location on iOS is something that not a lot of apps need, but after requiring it for my latest app, Scanula, I found that there isn't a good resource explaining how to do it properly. This is the first post in a series of planned posts going over a few of the tips, tricks, and common pitfalls I have found while working with iOS Share Sheets.
\nSharing on iOS is done using the Share Sheet, which is often opened via the "Action" icon (shown left). When tapping this, the user is presented with a Share Sheet, which provides various options, depending on the item being shared. In the blog post we'll be looking at location exclusively, but there are a various things that can be shared, from images, to URLs, to text files. The full list can be found in Apple's Documentation.
\nThere's not a whole lot of documentation that helps determine exactly how to use UIActivityViewController
and its associated classes, but the simplest way to use it would be:
// This code assumes:\n// - This is inside a subclass of `UIViewController`\n// - This is inside a class that contains a property called `shareBarButtonItem` of type `UIBarButtonItem`\n\nlet activityItems: [AnyObject] = [\n "A shared piece of text"\n];\n\nlet vc = UIActivityViewController(activityItems: activityItems, applicationActivities: nil)\n\n// If run on iPad, this is required\nvc.popoverPresentationController?.barButtonItem = shareBarButtonItem\n\npresentViewController(vc, animated: true, completion: nil)\n
\nThis would present a Share Sheet sharing "A shared piece of text". This would allow the user to share this text via various built-in applications, such as Messages, Mail, or Notes, via 3rd party apps, such as Dropbox or Facebook, or via AirDrop. On its own, this isn't particularly useful, but it's a start.
\nNote that UIActivityViewController
's designated initialiser takes in an array of AnyObject
. This may look like an open invitation to just pop anything in the array, but it's actually far from that. Even though it's smart enough to figure out what you want with simple things (such as text, as shown in the example), it cant infer what you want to send for all items.
For more complex items, items should conform to UIActivityItemSource
(something I hope to write another blog post on). However, in this case, simply using NSItemProvider
will help a lot.
The primary focus for this blog post is going to be sharing locations. So, without further adu, here's the meat to go with your potatoes.
\nWhen searching Google for "uiactivityviewcontroller share location", the top results (I've not checked all ~150,000) point to a very similar solution:
\nAddressBook
framework and using that to generate the VCard contents, crazy!NSURL
of the fileHere's my example code:
\n// Note: Don't use this code!\nfunc activityItems(latitude: Double, longitude: Double) -> [AnyObject]? {\n var items = [AnyObject]()\n\n let locationTitle = "Shared Location"\n\n let locationVCardString = [\n "BEGIN:VCARD",\n "VERSION:3.0",\n "PRODID:-//Joseph Duffy//Blog Post Example//EN",\n "N:;\\(locationTitle);;;",\n "FN:\\(locationTitle)",\n "item1.URL;type=pref:https://maps.apple.com/?ll=\\(latitude),\\(longitude)",\n "item1.X-ABLabel:map url",\n "END:VCARD"\n ].joinWithSeparator("\\n")\n\n guard let vCardData = locationVCardString.dataUsingEncoding(NSUTF8StringEncoding) else {\n return nil\n }\n\n let fileManager = NSFileManager.defaultManager()\n guard let cacheDirectory = try? fileManager.URLForDirectory(.CachesDirectory, inDomain: .UserDomainMask, appropriateForURL: nil, create: true) else {\n return nil\n }\n\n let fileLocation = cacheDirectory.URLByAppendingPathComponent("\\(latitude),\\(longitude).loc.vcf")\n vCardData.writeToURL(fileLocation, atomically: true)\n\n return [\n fileLocation\n ]\n}\n
\nWhile this does technically work for most use cases, when sharing via AirDrop the items is interpreted as a file (as it technically should). This has some unwanted side effects:
\nWhen trying to figure out the correct way of doing this I created a small app for debugging Share Sheet items (hopefully more on this in another blog post). This shows me that Apple's built-in Maps application does things a little differently by sharing:
\nNSURL
Doing this is fairly easy. Here's my code to do it:
\nfunc activityItems(latitude: Double, longitude: Double) -> [AnyObject]? {\n var items = [AnyObject]()\n\n let locationTitle = "Shared Location"\n let URLString = "https://maps.apple.com?ll=\\(latitude),\\(longitude)"\n\n if let url = NSURL(string: URLString) {\n items.append(url)\n }\n\n let locationVCardString = [\n "BEGIN:VCARD",\n "VERSION:3.0",\n "PRODID:-//Joseph Duffy//Blog Post Example//EN",\n "N:;\\(locationTitle);;;",\n "FN:\\(locationTitle)",\n "item1.URL;type=pref:\\(URLString)",\n "item1.X-ABLabel:map url",\n "END:VCARD"\n ].joinWithSeparator("\\n")\n\n guard let vCardData = locationVCardString.dataUsingEncoding(NSUTF8StringEncoding) else {\n return nil\n }\n\n let vCardActivity = NSItemProvider(item: vCardData, typeIdentifier: kUTTypeVCard as String)\n\n items.append(vCardActivity)\n\n items.append(locationTitle)\n\n return items\n}\n
\nThis doesn't require much more code, but has a few other added bonuses:
\nI've been doing a lot of work with Share Sheets lately, so if you've found this post useful and want to see more, check back soon, subscribe to the RSS feed for this blog, or follow me on Twitter.
", "url": "https://josephduffy.co.uk/posts/ios-share-sheets-the-proper-way-locations", "title": "iOS Share Sheets the Proper Way - Locations", "summary": "Sharing a location on iOS is something that not a lot of apps need, but after requiring it for my latest app, Scanula, I found that there isn't a good resource explaining how to do it properly. This is the first post in a series of planned posts going over a few of the tips, tricks, and common pitfalls I have found while working with iOS Share Sheets.
", "date_modified": "2016-03-07T16:39:46.000Z", "date_published": "2016-03-07T16:39:46.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/exploiting-university-security-for-my-own-convenience", "content_html": "This blog post covers an open-source timetable parsing project I released a couple of months ago. It is available at https://timetable.josephduffy.co.uk and the source is available on GitHub. The post won't go too in-depth on the technical side of the project, but rather the story of how I discovered it was possible.
\nSince starting my studies at the University of Huddersfield I've always wanted an easy way to see my timetable on my phone. The timetable available on the website isn't responsive and relies on POST data to display future weeks timetables, 2 things that don't work great on mobile, especially when the page is kept open in the background.
\nTo get around this I would manually add each of my lectures and practicals to my calendar. These events could be set as recurring, however they would often need removing on specific days (such as during holidays) or have different information on another date, such as a room change. All of this eventually led me think about the famous XKCD Automation comic, so I started work on a method of automating adding it to my calendar.
\nMy first idea for how to automate the process was to create a Google Chrome extension. To try and figure out if this was possible I loaded up my timetable.
\n\nHere I noticed that the URL didn't have any information about the timetable. My next thought is that my student number must be stored in a cookie. So I opened up the Developer Tools to inspect the request. To my surprise it was a POST request... with the student number as part of the form data.
\n\nBut surely it would be just be using that to validate that I was the user I said I was, right? I loaded up my favourite HTTP utility, DHC, and created a basic request to load my own timetable.
\n\nIt worked! Just to double check I messaged one of my friends explaining the situation and asking him for his student number. He sent me the number and, again, it worked! My first thought was that I was happy that I'd found a way that I might be able easily scrape the data I needed. My second thought was that it was a bit worrying that by only knowing someone's student number you could find out where someone was likely to be. Despite this I started thinking of how I could truly automate this.
\nSince it was so easy to access the data I thought it'd be a good idea to make the service available to others. Creating a Google Chrome extension would prevent some users from using the service and could make it a little harder to get it in to a user's calendar. The calendar would also not automatically update. The overall user experience would be worse.
\nGetting the current weeks timetable is easy, but what about future weeks? To figure this out I loaded up my timetable again and changed the value in the "Week beginning" dropdown.
\n\n
\nAs you can see, there's a bit more going on this time. However, having worked with ASP.NET before, I can see that it won't be too hard to make the request work. So I make another request:
\n\nNow we have ~~where people will be for the rest of the academic year~~ all of my future timetables!
\nAs per Atwood's Law, any application that can be written in JavaScript, will eventually be written in JavaScript, so naturally I turned towards Node.js.
\nI stuck with Express and found jsdom, a lovely framework for working with a DOM on the server. This would then allow me to pull the information and traverse the DOM on the server side. It might not handle errors too well but my timetable's markup doesn't appear to have changed since I started University, so it'll do.
\nSince I've got the DOM on the server-side to get future timetables I can simply take the value from the hidden input
s __VIEWSTATE
and __EVENTVALIDATION
and send them with the request. Simple!
Now came the tedious part: extracting the data and converting it to a format that calendar applications will understand. I've created single calendar events in the past, but never a full calendar with lots of extra fields, such as the VALARM
. Overall the iCalendar specification is rather long and complicated, but it's easy enough to focus on the parts needed for the project. Primarily I had to ensure that events would trigger at the right time, independent of time daylight saving time, which means adding ;TZID=Europe/London
to all event dates.
Add a couple of options for adding alarms prior to events and set the correct Content-Type
headers and you're set! There were a few kinks to work out but I've been using it for a few weeks now and love it.
Before I released the code or created the website I spoke to one of my lecturers to ask whether he knew if I was breaking any rules. Apparently he (along with other members of staff) has told the team responsible for the timetable website about the security issue and they've decided not to do anything about it and essentially ignore the problem. That's up to them, but personally I think it's a little creepy that someone could make a website where people can view anyone at the University's timetable. But who'd do that?
", "url": "https://josephduffy.co.uk/posts/exploiting-university-security-for-my-own-convenience", "title": "Exploiting University Security for My Own Convenience", "summary": "This blog post covers an open-source timetable parsing project I released a couple of months ago. It is available at https://timetable.josephduffy.co.uk and the source is available on GitHub. The post won't go too in-depth on the technical side of the project, but rather the story of how I discovered it was possible.
\nSince starting my studies at the University of Huddersfield I've always wanted an easy way to see my timetable on my phone. The timetable available on the website isn't responsive and relies on POST data to display future weeks timetables, 2 things that don't work great on mobile, especially when the page is kept open in the background.
\nTo get around this I would manually add each of my lectures and practicals to my calendar. These events could be set as recurring, however they would often need removing on specific days (such as during holidays) or have different information on another date, such as a room change. All of this eventually led me think about the famous XKCD Automation comic, so I started work on a method of automating adding it to my calendar.
", "date_modified": "2015-12-20T21:09:53.000Z", "date_published": "2015-12-20T21:09:53.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/gathered-1-0-1", "content_html": "Gathered 1.0.1 was released a couple of weeks ago and I wanted to write a quick blog post addressing part of the changelog:
\n\n\nRemoved the "Disable Device Sleep" option (at Apple's request)
\n
Without posting lots of useless code, the offending code was as follows:
\nUIApplication.sharedApplication().idleTimerDisabled = newValue\n
\nwhere newValue
is a Bool
.
This code was in Gathered 1.0, but upon submitting the 1.0.1 update I received the following message from Apple:
\n\n\nPLA 3.3.1
\nYour app uses public APIs in an unapproved manner, which does not comply with section 3.3.1 of the Apple Developer Program License Agreement.
\nSpecifically, this app does not meet the requirements of our UIApplication documentation. It would be appropriate to remove this app's use of idleTimerDisabled before resubmitting for review.
\nSince there is no accurate way of predicting how an API may be modified and what effects those modifications may have, Apple does not permit unapproved uses of public APIs in App Store apps.
\n
Note that the link to the Apple Developer Program License Agreement requires you to be logged in with a developer Apple ID. Looking up section 3.3.1, the relevant part is:
\n\n\nApplications may only use Documented APIs in the manner prescribed by Apple and\nmust not use or call any private APIs.
\n
Looking at the documentation for the idleTimerDisabled
property of UIApplication
, it states:
\n\nYou should set this property only if necessary and should be sure to reset it to
\nfalse
when the need no longer exists. Most apps should let the system turn off the screen when the idle timer elapses. This includes audio apps. With appropriate use of Audio Session Services, playback and recording proceed uninterrupted when the screen turns off. The only apps that should disable the idle timer are mapping apps, games, or programs where the app needs to continue displaying content when user interaction is minimal.
(emphasis is mine)
\nI assumed this was just something that was flagged internally by some sort of automated code scanner and submitted an appeal to the App Review Board, stating that I felt Gathered fell under the category of apps which "[need] to continue displaying content when user interaction is minimal." A few days later I received an email scheduling a phone call, in which I am told that the functionality would not be accepted. I was not give any more of a reason, other than "your app was found to be out of compliance with App Store Review Guidelines" and that I must remove the feature and resubmit my update.
\nAt this point I had no choice but to remove the feature. I removed it, and submitted an update with the changelog as shown on Gathered's changelog page. However, this was rejected due to the meta data with the following message:
\n\n\nWe noticed that your app's metadata includes the following information, which is not relevant to the application content and functionality:
\nRemoved the "Disable Device Sleep" option at Apple's request
\n
Ok, fair enough, I was being slightly cheeky. I update that line of the changelog to:
\n\n\nRemoved the "Disable Device Sleep" option
\n
and the update was approved.
\nSo, in summary, I'm sorry for the removal of the "Disable Device Sleep" feature, but I had no choice. This is also what caused the release of 1.0.1 to be delayed by 2 to 3 weeks. If you have any comments or feature requests for Gathered, either visit Gathered's feedback page or contact me on Twitter.
", "url": "https://josephduffy.co.uk/posts/gathered-1-0-1", "title": "Gathered 1.0.1 and the Disable Device Sleep Setting", "summary": "Gathered 1.0.1 was released a couple of weeks ago and I wanted to write a quick blog post addressing part of the changelog
", "date_modified": "2015-12-06T18:48:34.000Z", "date_published": "2015-12-06T18:48:34.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/its-a-duffy-thing", "content_html": "I recently released a major overhaul for this website. The old website used an old version Node.js and used Ghost to power the blog. I didn't find it very easy to maintain and wanted more flexibility. While the new website may not have the best design, I'm a lot happier with it overall. Along with the rewrite of the website itself, I also gave it a new name: It's a Duffy Thing. This was inspired by a shirt that my Dad bought me.
\nIn this blog post I want to go over a few of the technologies used to power the website. Partially so it's all together and in one place, but also as a kind of "behind the curtain" look at the website. If you want to dive in even further, take a look at the source code on GitHub.
\nThe new version of the website is built using Node.js, SASS, and Handlebars. There's no front-end JavaScript on the website other than a couple of little external scripts (such as Google Analytics), so there's nothing to discuss when it comes to front-end JavaScript.
\nI've liked the idea of Node.js for a while. For simple websites (such as this) that receive little traffic (such as this) and can be used to experiment with new technologies (such as this) I think it's great. My previous projects have used io.js, which was recently merged in to Node.js, but this project uses the latest stable version of Node at the time, version 4.2.2.
\nUsing Node has a few other advantages. Ones of those is being able to use gulp. There a lot of workflow automators out there now, and which is "best" seems to change fairly rapidly. I'm happy with my workflow using gulp so that's what I stick to. I use gulp to automate all of the following tasks:
\nThis is all done in a single gulp file. I've also added a watch
task so that changes made to SASS file automatically trigger a recompile. I'll likely extend this to the views directory, too, so that any CSS rules added or removed from the HTML will be add or removed from the compiled CSS.
When looking for a blogging engine I was torn between one which provided a lot of management and functionally out of the box (such as Ghost), or one which offered more of a basic set of scaffolding to work from. After trying Ghost for a while I opted for the simpler approach, which led me to Poet. Poet takes in a directory containing markdown files and converts them to HTML, pulling out some extra meta data from a JSON structure at the top of the file such as the URL slug, publish date, and tags.
\nI've really loved working with Poet. It doesn't get in the way and me choose how things should be. I actually override most of the routes and mainly use it as a markdown to HTML converter, but it works great for me.
\nI've used SASS on an off for a while but I really wanted to dive in a bit further this time. The design and layout for this website is fairly simplistic so I can get away with using Bootstrap and just adding a few additional styles. SASS is perfect for this, and as described in the gulp section, it ends up being really nice workflow which produces a fairly small file.
\nAlong with SASS, Handlebars is probably my favourite things that changed about my workflow when working with the web in recent years. I still find it hard to not put logic in to the Handlebar files (thanks, PHP), but I'm loving partials and inheritance. Magical!
\nHelmet is a great piece of middleware for Express which helps improve the security of a website. There's always more that can be done, but it's been very easy to add things like the Content Security Policy and setting the X-XSS-Protection
header.
I decided to open source the website for a couple of reasons:
\nI'm happy with my decision to open source the website. I can already see that I'm writing better commit messages and separating my commits up further.
\nI've got a few other post ideas (some of which are already 90% written), so there's going to be more activity on here soon. You can subscribe to the blog feed, follow me on Twitter, or simply check the website soon to see new posts.
", "url": "https://josephduffy.co.uk/posts/its-a-duffy-thing", "title": "It's a Duffy Thing", "summary": "I recently released a major overhaul for this website. The old website used an old version Node.js and used Ghost to power the blog. I didn't find it very easy to maintain and wanted more flexibility. While the new website may not have the best design, I'm a lot happier with it overall. Along with the rewrite of the website itself, I also gave it a new name: It's a Duffy Thing. This was inspired by a shirt that my Dad bought me.
", "date_modified": "2015-12-03T15:41:57.000Z", "date_published": "2015-12-03T15:41:57.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } }, { "id": "https://josephduffy.co.uk/posts/touch-id-on-the-lock-screen", "content_html": "Touch ID is a wonderful piece of technology, to the point where wouldn't buy an iOS device without it. It had many great uses, such as:
\nHowever, I wish to discuss the first of these: unlocking the device.
\nI do not currently own an iPhone 6s or iPhone 6s Plus so I have not been able to try out the new (claimed 2x faster) Touch ID. However, from my own experience using my iPhone 6 I can confirm the problem that some people feel they are having: the device unlocks too fast. Simply pressing the home button to check the time or a notification is often enough for the device to read the fingerprint and unlock. Personally, the majority of the time, unlocking the device is what I want to do. However, assuming that Touch ID continues to improve and its uses continue to grow I see a potential new way for it to work.
\nTouch ID currently works on the principal of:
\nThis is of course an over simplification, but it demonstrates what's happening. However, in the case of unlocking the device the action of "unlock the device" is assumed, which is not always correct. My proposal is to allow the user to pre-authorise an action from the lock screen. The user could then perform any of the following actions without having to touch the Touch ID sensor:
\nThis would obviously be a confusing change for some users, and is something would likely only appeal to the nerds that care about the small time save/loss. For this reason I would personally set it as an option that is off by default (or mentioned during setup) and have a visual indicator of what's happening.
\n\nLockGlyph (shown above) is a tweak available for iOS 8 that adds the Apple Pay animation to the lock screen when unlocking. If instead of fully unlocking the device when authentication the checkmark were to stay to indicate authentication and acted as the pre-authorisation for the next action. To try and keep the action of unlocking quick the checkmark could also be tapped to unlock the device, as well as the classic slide to unlock. It may even be possible for Apple to implement some form of check so that when the user may wish to interact with the lock screen (such as a notification or music controls being shown) this options is turned on, but in other situations the auto-unlock would only happen based on the user preferences. To summarise, the interaction would go a little like this:
\nThat's a simple little idea I thought about recently. I'm no UX Designer so this may be an awful idea, but I feel I would personally benefit from it. Either way, I'd love to hear your thoughts. The best way to contact me would be via Twitter.
", "url": "https://josephduffy.co.uk/posts/touch-id-on-the-lock-screen", "title": "Touch ID on the Lock Screen", "summary": "Touch ID is a wonderful piece of technology, to the point where wouldn't buy an iOS device without it. It had many great uses, such as:
\nHowever, I wish to discuss the first of these: unlocking the device.
", "date_modified": "2015-10-03T11:34:24.000Z", "date_published": "2015-10-03T11:34:24.000Z", "author": { "name": "Joseph Duffy", "url": "https://josephduffy.co.uk/" } } ] }