At this year’s Augmented World Expo (AWE) in Long Beach, California, I watched every talk and discussion online about the Apple Vision Pro. From these sessions, I identified some common threads from developers about the challenges and opportunities of developing for this new platform. I will be continuing this conversation at AWEAsia in Singapore this August. For more information, visit AWEAsia or my website at https://erichawkinson.com.

Apple has always had a knack for creating buzz around its products, and the Vision Pro is no exception. Touted as a groundbreaking device set to revolutionize spatial computing, the Vision Pro entered the market with high expectations and an even higher price tag. However, as the initial excitement wanes and sales projections are halved, it’s time to look beyond the marketing glitz and examine the real challenges faced by developers working with this new hardware.

Despite the fanfare, the Apple Vision Pro’s journey into the hands of developers has been anything but smooth. The device, with its eye-watering $3,500 price tag, promised a new era of mixed reality but has instead highlighted the growing pains of pioneering in this space. Developers have had to grapple with various issues that Apple’s glossy ads conveniently overlook. Make no mistake though, it is an amazing piece of hardware and a feat of engineering and should be celebrated, but the overall business models and ever higher walls in the garden are showing signs of too much for developers. If developers find it harder to thrive in this ecosystem, it could slow or hurt progress in the future of XR.

This struggle isn’t unique to those developing for Apple Vision Pro; it mirrors broader challenges faced by developers across various platforms, including the web and Meta Quest. In contrast, platforms like Meta Quest, which don’t necessarily ’embrace’ WebXR and open standards, offer developers more flexibility and fewer constraints, allowing for a more diverse and dynamic development environment. A wonderful example of this is SideQuest, which Meta supports with App Lab and while makes users jump though a ton of hoops, at least makes a legit way to have a thriving 3rd party ecosystem for software on the Quest. This disparity highlights a critical issue: while Apple’s tightly controlled ecosystem aims to ensure quality and security, it often stifles innovation and restricts developers’ ability to connect directly with their users.

The intent to limit the use of the open when on AVP is a clear indication of Apple’s stance. Apple’s support for WebXR has been lukewarm, marked by limited concrete actions despite recent statements indicating a willingness to embrace the technology. The tech giant continues to prioritize proprietary media formats and standards, which stands in stark contrast to the principles of an open and immersive web. This approach has created significant friction within the development community, particularly among those who advocate for interoperability and open standards. In my article “The Upgrade No Developer Asked For,” I explore the broader implications of Apple’s recent policies on developers, particularly in the context of their 27% commission on external payment methods. This move, perceived as an extension of the infamous ‘Apple Tax,’ underscores the company’s ongoing efforts to maintain control over its ecosystem. These policies not only impose financial burdens on developers but also raise critical questions about market fairness and Apple’s influence over its App Store. This has become such an issue that the EU has begun to intervene with the Digital Markets Act.

Introducing Apple’s New iTax: The Upgrade No Developer Asked For

For educational developers like myself, this tightening grip presents additional challenges, exacerbating issues of ‘lock-in’ with proprietary systems. The need for open, accessible platforms is crucial, especially in education, where the primary goal is to enhance learning and accessibility.

Since the announcement of the Apple Vision Pro, Apple has meticulously controlled the narrative surrounding the device. Over the past year, the company has avoided using terms like VR or XR, instructing developers to adopt the same language, thereby shaping the discourse to fit its strategic vision. This tight grip on the narrative has limited independent evaluations and critical discussions, which are essential for a balanced understanding of any new technology. In my article “A Fancy Term for ‘We’re Not Calling It XR’,” I delve into how Apple’s insistence on terms like “spatial computing” over more conventional terminology may complicate efforts to educate the public about these technologies. The shift in terminology aims to differentiate Apple’s product in a crowded market, aligning it with a more inclusive concept that transcends traditional VR and AR frameworks. However, this rebranding strategy also risks creating confusion among consumers and developers, especially those who are accustomed to established industry terms. For further reading, refer to the full article on my website. However, the tide is beginning to turn.

Apple’s ‘Spatial Computing’: A Fancy Term for ‘We’re Not Calling It XR’

At this year’s Augmented World Expo (AWE) in Long Beach, California, industry leaders and developers have come together to share their unfiltered experiences with the Vision Pro. AWE, renowned for being the premier event where XR business thrives, has provided a platform for these voices to be heard. The expo, which featured over 300 exhibitors and attracted a global network of spatial computing professionals, served as a stage for open dialogue and candid discussions about the challenges and opportunities of developing for Apple’s latest device.

I watched everything from AWE about the AVP and found some common threads, to offer some of the first cohesive unbiased accounts of what it’s like to develop for the Vision Pro. This event, with the Vision Pro as a central topic of discussion, highlights the experiences and insights of developers who are navigating the complexities of Apple’s new platform.

The Enthusiastic Evangelist: Bert Nepveu

Bert Nepveu, an XR veteran and former Apple insider, brings to light the substantial challenges that accompany the Vision Pro development. Despite his excitement for the future of spatial computing, he does not shy away from highlighting the critical issues:

1. Cultural Challenges at Apple: Nepveu highlights the secrecy and high-stakes environment at Apple, likening it to working for the CIA. This culture can be stifling for developers used to more open and collaborative settings.
2. Over-Engineering Issues: The Vision Pro is described as an “over-engineered” device. While it looks great with its aluminum and glass design, this focus on aesthetics has resulted in a device that is too heavy and cumbersome for prolonged use.
3. Limited Documentation and Support: Nepveu points out the lack of comprehensive documentation and support for developers, making it difficult to navigate the new interaction paradigms required for the Vision Pro.
4. Privacy Constraints: Apple’s strict privacy policies, while commendable, limit developers’ access to key features like the main cameras, hampering their ability to create innovative applications.
5. Market Accessibility: With the Vision Pro currently available only in the US, developers are frustrated by the limited market reach, which restricts their ability to test and deploy applications globally.

The Pragmatic Product Designer: Daniel Marqusee

Daniel Marqusee, a seasoned product designer, offers a pragmatic perspective on the challenges of designing for Vision Pro. His extensive experience in spatial design highlights several critical pain points:

1. Transition from 2D to 3D Design: Marqusee emphasizes the difficulty for designers accustomed to 2D interfaces to adapt to 3D spatial design. Many designers lack the skills and guidelines to work with depth, making the transition challenging.
2. Inadequate Design Standards: The current design standards for Vision OS are described as “smashed together,” indicating a lack of cohesive and comprehensive guidelines for developers to follow. This contributes to confusion and inconsistency in design practices.
3. Ergonomic and Usability Issues: Marqusee criticizes the Vision Pro’s reliance on hand-based interactions, describing them as an “ergonomic nightmare.” This method, while innovative, can be limiting and uncomfortable for users.
4. Privacy and Interoperability Limitations: Apple’s strict privacy policies and prioritization of their own formats over interoperability create barriers for developers. This limits the ability to create seamless and integrated experiences across different platforms.
5. Monetization and App Model Issues: The traditional app-to-app model is seen as outdated and ineffective for XR. Marqusee advocates for a shift towards more dynamic, web-based applications that can adapt and integrate more fluidly.

The Veteran Developer: James Ashley

James Ashley, with extensive experience in Unity and native Vision OS development, provides insights into the technical challenges faced by developers:

1. High Initial Costs: Ashley highlights the significant financial investment required to develop for Vision OS, including the need for a Mac with a silicon chip and the $2,000 annual Unity Pro license, which can be prohibitive for indie developers and small studios.
2. Learning Curve: Transitioning from Unity to native Vision OS development involves learning new tools and languages, such as Swift, SwiftUI, and Reality Composer Pro. This can be a daunting task for developers accustomed to Unity.
3. Documentation and Support: While Apple provides extensive documentation and sample code, Ashley notes that Unity’s resources are more limited. Developers often have to rely on community forums for support, which can be inconsistent.
4. Performance Considerations: There is a perception that native development may offer better performance and access to new features and APIs more quickly than Unity. However, this advantage is not always clearly defined, and concrete performance benefits are often based on hunches rather than solid evidence.
5. Complex Development Ecosystem: The Vision OS development environment is described as complex, with a need to understand a new set of tools and frameworks. This complexity can add friction to the development process, especially for teams used to the Unity ecosystem.

The Vision OS Expert: Andy Roth

Andy Roth, a seasoned Vision OS engineer, sheds light on the updates and enhancements in Vision OS 2, providing a comprehensive look at the platform’s evolution and the new tools available to developers:

1. Incomplete Development Environment: Roth notes that Vision OS is still evolving, with many features and APIs being recently introduced or updated. This ongoing development means that the platform is not fully stable, requiring developers to constantly adapt to new changes and updates.
2. Complexity of New Tools and APIs: The introduction of new tools like Reality Kit 4, Reality Composer Pro 2, and various Enterprise APIs adds layers of complexity to the development process. Developers need to learn and integrate these tools effectively to leverage the platform’s capabilities.
3. Gestural Interaction Challenges: New gesture controls, such as using hand gestures to access the home screen or control volume, require significant adjustment for both developers and users. Ensuring these gestures work smoothly and intuitively can be a challenge.
4. Ergonomic and Usability Issues: Similar to previous insights, Roth points out that the Vision Pro’s reliance on hand-based interactions and the visibility of hands in dark environments can affect the immersive experience. Developers need to manage these ergonomic issues carefully.
5. Enterprise Restrictions: The new Enterprise APIs, while powerful, come with additional entitlements and distribution restrictions. These limitations can be a hurdle for developers aiming to create applications for broader markets.

The WebXR Advocate: Dulce Baerga

Dulce Baerga, a staunch advocate for WebXR, highlights the potential and challenges of web-based XR development on the Vision Pro:

1. Privacy and Security Concerns: One of the significant limitations is the restricted access to AR capabilities due to Apple’s stringent privacy policies. This limitation curtails the potential of true AR experiences on the Vision Pro via WebXR.
2. Interoperability Issues: While WebXR promises cross-platform compatibility, achieving seamless interoperability across different devices and browsers remains a challenge. The inconsistent implementation of WebXR standards can lead to fragmented user experiences.
3. Distribution and Discoverability: The lack of established distribution channels for WebXR content poses a significant challenge. Unlike app stores, there are no centralized platforms for discovering and distributing WebXR experiences, limiting their reach and impact.
4. Performance Optimization: Ensuring optimal performance of WebXR experiences on various hardware configurations is complex. Developers must balance the richness of the experience with the performance limitations of web technologies and hardware variations.

Conclusion: Navigating the Vision Pro Landscape

he Apple Vision Pro, despite its groundbreaking potential, is beset with numerous challenges that developers must navigate. From hardware design issues and high development costs to fragmented user experiences and stringent privacy policies, the road to creating successful applications for Vision Pro is fraught with obstacles. However, these challenges also present opportunities for innovation and improvement.

As the platform matures and the development ecosystem evolves, the Vision Pro has the potential to redefine the boundaries of spatial computing. Developers, armed with critical insights and a willingness to adapt, can play a pivotal role in shaping the future of this exciting technology. Apple’s recent stance on WebXR and its broader approach to controlling the developer ecosystem highlights a significant tension between its business model and the needs of developers. This conflict is particularly evident in the context of the Apple Vision Pro (AVP) and the immersive web.

Developers are increasingly seeking ways to establish direct relationships with their customers, pulling users out of tightly controlled app ecosystems. In contrast, Apple aims to maintain its position as the intermediary, a strategy that supports its business model but often hampers developers’ ability to innovate freely and maximize their revenue. While it is Apple’s right and fiduciary responsibility to protect its business interests, I contend that the balance is starting to show signs of wear within the developer community, particularly from my perspective as an educational technology developer. This tension could ultimately stifle creativity and hinder the progress of educational advancements if developers like myself find ourselves overly constrained by Apple’s ecosystem.

As discussed in “Native Development for Apple Vision Pro” by James Ashley, Apple’s tools and platforms are designed to keep developers within the Apple ecosystem. Ashley notes, “if you’re building something for an Apple platform, it should look like an Apple app,” emphasizing the company’s desire to ensure uniformity and control over the user experience. This control extends to the functionalities that developers can access and the performance they can extract from the device, which often lags behind when using third-party engines like Unity. Andy Roth, in “visionOS Updates,” highlighted how Apple’s proprietary approaches impact the developer community. He mentioned improvements and new features in visionOS but also pointed out limitations, such as the lack of full AR support due to privacy concerns with camera access. These constraints are indicative of Apple’s broader strategy to keep tight reins on how its devices are used, prioritizing their ecosystem’s integrity over developer flexibility. Moreover, Dulce Baerga in “Web Development for Apple Vision Pro” underscores the potential of WebXR to provide a more open and interoperable platform. Baerga, a staunch advocate for WebXR, notes that despite some improvements, Apple’s support remains limited. This lack of robust support for WebXR on Apple devices forces developers to navigate complex and often restrictive environments, stifling innovation and making it difficult to deliver seamless, cross-platform experiences.

Apple’s 27% commission on external payments, as I discussed in “The Upgrade No Developer Asked For,” further illustrates this tension. By imposing high fees on transactions initiated outside the App Store, Apple effectively discourages developers from seeking alternative payment methods that could offer more competitive pricing to consumers. This policy, while legally compliant, undermines the spirit of fostering a more competitive marketplace. In essence, Apple’s business model is designed to keep users within its ecosystem, creating a controlled environment that benefits the company through commissions and proprietary standards. However, for developers, particularly those in the educational sector, these constraints pose significant challenges. The need for open, accessible platforms is critical to enhancing learning and providing valuable resources without being unduly hindered by restrictive policies.

As the landscape of digital marketplaces continues to evolve, the clash between Apple’s desire for control and developers’ need for autonomy will likely intensify. Developers must navigate this complex terrain, balancing the benefits of Apple’s ecosystem against the limitations it imposes.

In short, we need more support for the open web for so many reasons, not the least is to support a developer community but also to have more interoperability and accessible content/experiences. This is KEY for educators like me.

About the Author

Eric Hawkinson
Learning Futurist

erichawkinson.com

Eric Hawkinson is a Learning Futurist at Kyoto University of Foreign Studies, where he focuses on the integration of technology into education. Specializing in the creation of immersive learning environments, Eric employs augmented and virtual reality to enhance learning outcomes. He is an advocate for digital literacy and privacy, promoting open access to information and ethical technology practices. Outside his academic role, Eric is engaged in public outreach and professional development. He has established immersive learning labs, designed online courses, and advised on technology strategies across various sectors. His professional designations include Adobe Education Leader, Google for Education Certified Innovator, and Microsoft Innovative Expert. Eric’s notable projects, such as AR experiences for TEDxKyoto and WebVR for Model United Nations, reflect his commitment to using advanced technologies for global education and collaboration. Eric is dedicated to exploring the challenges and opportunities presented by emerging technologies, contributing significantly to the evolution of educational practices.

Roles
Professor – Kyoto University of Foreign Studies
Research Coordinator – MAVR Research Group
Founder – Together Learning
Developer – Reality Labo
Community Leader – Team Teachers
Co-Chair – World Immersive Learning Labs

I’m thrilled to introduce an exciting addition to our curriculum at the Kyoto University of Foreign Studies, “Digital Citizenship and Engagement,” designed by Professor Eric Hawkinson. This innovative course is set to debut this fall and will welcome its first cohort of approximately 250 students.

In conjunction with the launch of this course, we are excited to share the official song of Digital Citizenship at Together Learning, titled “Digital Citizens.” This song encapsulates the spirit of our digital age, emphasizing the importance of navigating the online world with wisdom, ethics, and respect. As we embrace technologies that reshape every aspect of our lives—from work and love to learning and health—understanding our role and responsibilities online has never been more critical.

“Digital Citizens” serves as a melodious reminder to maintain an open and curious mind while securing our digital footprints. It celebrates the blend of our real and online lives, urging us to create a better world through the skills digital worlds we all share. We invite you to listen to this song as a prelude to the enriching journey that lies ahead in exploring the deep and intricate relationships between technology and humanity in all aspects of life.