Getting Started With ARKit If You Already Know Swift
ARKit Apple Developer tools are not particularly difficult once you get past the initial setup phase. The framework has been around since 2017 and has matured significantly since then. I started working with it back when version 1.0 was brand new and the documentation was rough around the edges. Things are much better now, but there are still a few things nobody really tells you until you hit them head-on. You need Xcode 15 or later, an iPhone or iPad with an A12 chip or newer, and a paid Apple Developer account if you want to test anything on actual hardware. The simulator handles basic scene rendering but it cannot do motion tracking. That means face tracking, plane detection, and hit testing simply do not work in the simulator. You will waste hours debugging what looks like a code error before realizing the simulator is the problem.
ARkit Apple Developer: Setting Up Your First Project
Create a new Xcode project and choose Augmented Reality under the App Template section. Make sure the Content Technology is set to SceneKit unless you are specifically building something with Metal, in which case use RealityKit. The template gives you a basic ARSCNView and a preconfigured session. It is not production-ready code but it demonstrates the core pieces working together. The key configuration lives in your ARWorldTrackingSessionConfiguration. You need to enable plane detection and high-performance mode. Without high-performance mode, ARKit downsamples your camera feed and you lose tracking quality almost immediately. I once spent an afternoon chasing jittery anchor placement only to discover the performance preset was left at medium. Setting it to high fixed the issue in under a minute. Here is what that configuration looks like in practice:
let config = ARWorldTrackingSessionConfiguration()
config.planeDetection = [.horizontal, .vertical]
config.environmentTexturing = .automatic
config.isLightEstimationEnabled = true That last line about light estimation is important. By default ARKit does not estimate the lighting in your environment. Objects you place will look like they are floating because they do not react to the real world's light direction or intensity. Enabling it gives you diffuse and specular estimates you can apply to your virtual objects. It is not perfect but it is good enough for most use cases.
Get the Full Details
Common Pitfalls That Will Waste Your Time
The most frequent problem developers run into is anchor drift. This happens when ARKit loses track of a surface and your virtual object slowly floats away from where it should be. It is usually caused by poor lighting, reflective surfaces, or too little visual texture on the plane you are detecting. A white tiled floor with uniform grout lines is essentially invisible to ARKit's feature detection. I have seen projects fail completely because the client chose a location with those exact conditions. The workaround was to add artificial markers or switch to LiDAR-based scanning on supported devices. Another issue is session interruption handling. Your AR session will get interrupted if a phone call comes in, the user switches apps, or the device gets too hot. If you do not properly save and restore your anchors when the session resumes, your users will see their placed objects vanish. The standard approach is to store anchor identifiers in an array and re-add them in sessionWasInterrupted and sessionResumed. Do not skip this. Users expect persistence. Face tracking has its own set of quirks. ARKit's face blendshape system returns 52 standard targets plus custom shapes on newer devices. The data comes in as unitless values between zero and one but they are not probabilities. A value of 0.5 does not mean fifty percent of something is happening. It is just a normalized position along that blendshape's axis. Mapping these to 3D model vertices requires a proper rig that supports the ARKit skeleton structure. I learned this the hard way when I tried to bind face tracking data to a generic humanoid rig and ended up with a completely misaligned mesh.
Performance Considerations You Should Not Ignore
Running ARKit at full resolution drains battery fast and generates significant heat. On an iPhone 13 Pro, enabling both plane detection and light estimation at 60fps will drop your frame rate to around 30fps after about twenty minutes due to thermal throttling. The device will also warn the user about temperature. This is not a software bug. It is hardware behavior. To mitigate this, consider reducing plane detection to horizontal-only if you do not need vertical surfaces. Disable light estimation if your visual style does not require it. Use ARKit's built-in collision detection sparingly. Custom physics implementations using PhysicsKit or a dedicated engine tend to perform better than relying on ARKit's basic capabilities. The framework is designed for accuracy first, performance second. RealityKit is the recommended path going forward. It replaced much of the older SceneKit-based AR workflow and provides better rendering performance out of the box. If you are starting a new project in 2025, do not begin with SceneKit. RealityKit handles occlusion, shadow casting, and material rendering more efficiently. Migration from existing SceneKit projects is straightforward but not automatic. You will need to rewrite your scene graph setup.
Where to Get the Tools
Everything you need is already included with Xcode from the Mac App Store. There is no separate download for ARKit. You will also want to install the AR Quick Look framework if you plan to share your experiences through SharePlay or standalone .usdz files. Those files can be opened directly in Safari without any app installation. This is useful for retail and e-commerce applications where you want users to preview objects in their space immediately. The official Apple developer documentation is reasonably good now. Start with the ARKit documentation page, then move to the RealityKit guides. The sample projects in Xcode under File New Project are worth examining even if you do not use them directly. They show patterns that are not obvious from the API reference alone. One thing I wish I had known earlier: ARKit works well with SwiftUI now. The ARViewContainer wrapper makes it easy to embed AR experiences in a SwiftUI hierarchy. The old ARCone and AREffectViewController approaches are legacy. If you are building a modern app, use SwiftUI with ARView. It reduces boilerplate code significantly and integrates better with the rest of the iOS ecosystem.
