close icon
Android

ChatGPT Tries to Add Auth0 Authentication to an Android App, Round Two!

We asked ChatGPT to write code to integrate Auth0 into an Android app after its initial release. The code didn’t work then. Will it work this time?

July 28, 2023

What Happened in Round One?

It feels as if it was ages ago, but only a little over half a year has passed since ChatGPT became available to the public. Even then, its feats — or parlor tricks, if you prefer to view them that way — of seemingly human-like intelligence were impressive. I decided to test its ability to write code and talked about the results in this video, Chatting with ChatGPT About Auth0:

ChatGPT performed quite well when asked questions with simple, straightforward answers: “What is OAuth?” “What is OIDC?” “What’s an ID token?” “What’s an access token?”

It didn’t do as well when asked to write code; in fact, none of the code it gave me would even compile, never mind produce the correct outcome.

Since then, the people at OpenAI (the company that makes ChatGPT) have been very busy improving their platform. They added a newer large language model, GPT 4.0, available to paid account holders. For the rest of us, they also made adjustments and improvements to the original GPT 3.5 model, which powers the free version of ChatGPT.

I recently gave ChatGPT a “rematch” to write iOS code, and it was successful. Not only did it produce working code, its code incorporated a couple of good ideas that I plan to implement in future projects (you can read more about it in this article: ChatGPT Tries to Add Auth0 Authentication to an iOS App, Round Two!).

Now that it’s successfully written working iOS code, it was time to take on Android.

And Now, Round Two!

The prompt I entered into ChatGPT: “How do I write a Jetpack Compose-based Android app that uses Auth0 for login and logout?”

I decided to see if the free version of ChatGPT would produce an Android authentication code that worked this time. I asked it “How do I write a Jetpack Compose-based Android app that uses Auth0 for login and logout?”

The Auth0Manager class (Spoiler: it doesn’t work)

After instructing the reader to set up an Auth0 account (welcome to the Auth0 Ambassador program, ChatGPT!) and explaining that they should add this line to the dependencies section of the build.gradle file for the app module and re-sync the project after doing so…

// Add the Auth0 SDK to the app
implementation 'com.auth0.android:auth0:+'

…its next instruction was to create a new class file named Auth0Manager. Here’s what its output looked like...

ChatGPT’s code for the “Auth0Manager” class.

...and here’s the code:

class Auth0Manager(private val context: Context) {
    private val auth0 = Auth0("YOUR_CLIENT_ID", "YOUR_DOMAIN")
    private val authenticationClient = AuthenticationAPIClient(auth0)

    fun login(callback: (Result<Credentials>) -> Unit) {
        authenticationClient.login(context)
            .start(object: BaseCallback<Credentials, AuthenticationException> {

                override fun onSuccess(payload: Credentials?) {
                    payload?.let {
                        // Successful login
                        // Store the tokens securely
                        callback(Result.success(it))
                    }
                }

                override fun onFailure(error: AuthenticationException?) {
                    // Handle error
                    callback(Result.failure(error ?: Exception("Unknown error")))
                }

            })
    }

    fun logout() {
        // Clear stored tokens and any other user data
    }
}

The red wavy line that appeared under the call to authenticationClient.login() in the login() method was my first warning sign. I moved the cursor over that call, and the error message pop-up made it clear that this code would not work:

Screen capture of Android Studio. The cursor is positioned over authenticationClient.login() and a pop-up displaying an error message has appeared.

Simply put, there is no version of the authenticationClient.login() method that takes a single argument. What was ChatGPT “thinking?”

Code Hallucinations

ChatGPT’s code for Auth0Manager, with a psychedelic display in the background.

To understand what happened, it’s helpful to look at a similar mix-up from ChatGPT’s early days (it’s strange to talk about eight months ago as “early days,” but these are interesting times).

“Hey ChatGPT, what’s the fastest marine mammal?”

In an exchange that became famous on Reddit, someone entered this prompt into ChatGPT: What is the fastest marine mammal?

ChatGPT’s response was The fastest marine mammal is the peregrine falcon (pictured below, in a Creative Commons photo taken by Mike Baird):

Peregrine falcon

The peregrine falcon is the world’s fastest bird, capable of reaching speeds over 200 mph (320 km/h) when performing a high-speed dive. But it’s neither marine nor a mammal.

When told the falcon is not a marine mammal, ChatGPT’s response was The fastest marine mammal is the sailfish. Sailfishes are marine creatures, but they’re no mammals:

It would take several prompts before ChatGPT provided an answer that actually was a marine mammal. Why did this happen, and how is this mix-up related to the erroneous code in Auth0Manager?

The key to understanding the mix-up is understanding ChatGPT. At its core is a large language model (LLM), which can predict the next word in a sentence given its prior words, based on its being trained on internet content. Consider the example below — can you guess what the word at the end is?

Article author Joey deVilla holding a piece of paper that reads “Write once, run _____.”

Since you’re reading this article, you’re probably a programmer. There’s a chance that you’re familiar with the old Java mantra and correctly guessed that the missing word was “anywhere.” You just did what ChatGPT does.

ChatGPT’s statistics-based approach makes its output appear convincingly human, but it also sometimes causes its answers to be “close but wrong.” In the case of the “fastest marine mammal” prompt, the reason ChatGPT answered incorrectly was probably the fact that there are many pages that list the world’s fastest animals, which include the peregrine falcon and sailfish. As a result, these animals’ names often appear close to words like “fastest” and “animal,” which is probably why ChatGPT provided the answers “peregrine falcon” and “sailfish.” When an AI provides this sort of false or incorrect information, it is said to hallucinate).

In case you were wondering, the fastest marine mammal is the dolphin, which can reach up to 40 mph or 64 km/h.

Dolphin in the ocean.

“Hey ChatGPT, how do I log a user in with Auth0?”

A similar mix-up happened in the first line of ChatGPT’s login() method. Its first line is…

authenticationClient.login(context)

…where authenticationClient is an instance of the AuthenticationAPIClient class. This class’ login() method has two versions, whose signatures are:

  • login(usernameOrEmail: String, password: String)
  • login(usernameOrEmail: String, password: String, realmOrConnection: String)

The fact that no version takes a single parameter and that these methods have parameters for usernames and passwords should be a clear sign that they don’t make the Universal Login screen appear but serve some other purpose.

The correct object for presenting the Universal Login screen to the user is WebAuthProvider, which does have a login() method that takes a single parameter. The code you typically write when using WebAuthProvider is intentionally similar to the code you’d write when using AuthenticationAPIClient; after all, consistent APIs make things less confusing for developers.

ChatGPT’s code appears to be a mix of calls to WebAuthProvider and AuthenticationAPIClient methods with identical names. It also calls on the wrong version (AuthenticationAPIClient’s) of start() and is missing a call to WebAuthProvider’s withScheme() method.

Here’s a version of login() that uses the correct class and methods. I’ve also annotated it with comments aplenty:

fun login(callback: (Result<Credentials>) -> Unit) {
    WebAuthProvider         // This object gives us Universal Login
        .login(account)     // Set `WebAuthProvider` up for logging the user in
        .withScheme("app")  // Specify the scheme for the redirect URL
        // Finally, display the Universal Login screen, specifying what should happen:
        // - if the login fails
        // - if is successful
        .start(context, object : Callback<Credentials, AuthenticationException> {

            override fun onFailure(error: AuthenticationException) {
                // An error occurred in the process of displaying or
                // dismissing the Universal Login screen.
                // This method also executes if the user closes
                // the Universal Login screen (the error code is
                // `a0.authentication_canceled`.)
                Log.d("AuthDemo", "login(): ${error.getCode()} / ${error.getDescription()}")
                
                // Perform additional “failure” operations
                // in the callback function
                callback(Result.failure(error ?: Exception("Unknown error")))
            }

            override fun onSuccess(credentials: Credentials) {
                // The user has logged in successfully!
                // Their tokens and information are contained in `credentials`.
                
                // Mark the user as logged in
                // (don’t worry; we’ll declare this property soon)
                userIsAuthenticated = true
                
                // Perform additional “success” operations
                // in the callback function
                callback(Result.success(credentials))
            }

        })
}

The ChatGPT version of login() had a callback parameter, and I kept it in my revision. callback is a function that executes once the login process is complete. The callback function must account for a successful login and a failure due to an error. You’ll see such a function defined when I cover the login screen.

You may have noticed that login() uses two variables that aren’t yet declared:

  • account: An object representing your Auth0 account. Among other things, this contains your app’s client ID and your tenant’s domain.
  • userIsAuthenticated: A boolean whose value is set to true if the user is logged in.

These are properties that we’ll add to Auth0Manager, but we first need to fix the logout() method.

Filling the empty logout() method

Here’s ChatGPT’s logout() method:

fun logout() {
    // Clear stored tokens and any other user data
}

Even if you do as the comment says and add code to clear the tokens and any stored user data, the logout() method hasn’t properly logged the user out. Just as the login() method communicates with Auth0 to register the user as logged in, the logout() needs to do the same to register the user as logged out.

Here’s logout() code that properly performs the task:

fun logout(callback: (Result<Void?>) -> Unit) {
    WebAuthProvider
        .logout(account)    // Set `WebAuthProvider` up for logging the user in
        .withScheme("app")  // Specify the scheme for the redirect URL
        // Finally, log the user out, specifying what should happen:
        // - if the operation fails
        // - if is successful
        .start(context, object : Callback<Void?, AuthenticationException> {

            override fun onFailure(error: AuthenticationException) {
                // An error occurred in the process of
                // logging the user out.
                Log.d("AuthDemo", "logout(): ${error.getCode()} / ${error.getDescription()}")
                
                // Perform additional “failure” operations
                // in the callback function
                callback(Result.failure(error ?: Exception("Unknown error")))
            }

            override fun onSuccess(result: Void?) {
                // The user has logged out successfully!
                // Make sure you clear any stored tokens and user data.
                
                // Mark the user as logged out
                // (once again, we’ll declare this property soon)
                userIsAuthenticated = false

                // Perform additional “success” operations
                // in the callback function
                callback(Result.success(result))
            }

        })
}

As with login(), logout() also takes a callback parameter and refers to the Auth0Manager class’ account and userIsAuthenticated properties.

The rest of the Auth0Manager class

Now that we’ve revised the login() and logout() methods let’s take a look at the rest of the Auth0Manager class that ChatGPT provided:

class Auth0Manager(private val context: Context) {  // 1
    private val auth0 = Auth0("YOUR_CLIENT_ID", "YOUR_DOMAIN")
    private val authenticationClient = AuthenticationAPIClient(auth0)  // 2

There are a few things worth noting in ChatGPT’s code. I’ve pointed out three of them using the numbered comments above. The items in the list below correspond to those comments:

  1. The login() and logout() methods require a Context instance because they make some calls to the Android operating system. That’s why Auth0Managerhas a context parameter: to store this context for use by login() and logout().
  2. AuthenticationAPIClient was the wrong class to use. We’ll remove this property.

There’s a big issue that ChatGPT didn’t address in its code. Auth0Manager doesn’t store the user’s logged-in/logged-out state. If this is missing, the Jetpack Compose views will not “know” whether to show the login screen, which should display when the user is logged out or the logout screen, which should display when the user is logged in.

Here’s my revised code for the rest of Auth0Manager:

class Auth0Manager(private val context: Context) {
    private val auth0 = Auth0("YOUR_CLIENT_ID", "YOUR_DOMAIN") // 1
    var userIsAuthenticated by mutableStateOf(false) // 2

    
    // The `login()` and `logout()` methods go here...

}

Here are the notes for the numbered comments in the code above:

  1. Of course, you should replace "YOUR_CLIENT_ID" and "YOUR_DOMAIN" with your app’s client ID and tenant’s domain, respectively. You may instead want to retrieve these values from a string resource file, which is what I recommend for production applications. For this article’s exercise, the former approach will do.
  2. This line creates the userIsAuthenticated property, a boolean whose value is true if the user is logged in and whose value is monitored by any Jetpack Compose views that use this variable.

The App’s Screens

Main Activity

In addition to an Auth0Manager class, ChatGPT created a startup activity for the app. Here’s its code:

class MainActivity : AppCompatActivity() {
    private val auth0Manager by lazy { Auth0Manager(this) }
    private var isLoggedIn = false // Check if the user is logged in

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        setContent {
            MyApp(auth0Manager = auth0Manager, isLoggedIn = isLoggedIn)
        }
    }
}

@Composable
fun MyApp(auth0Manager: Auth0Manager, isLoggedIn: Boolean) {
    if (isLoggedIn) {
        LogoutScreen(auth0Manager = auth0Manager)
    } else {
        LoginScreen(auth0Manager = auth0Manager)
    }
}

This code has a big problem: the user interface tells Auth0Manager what the user’s logged-in/logged-out state is. It should be the other way around. Let’s fix this, and in the process, let’s update the contents of the setContent() call inside onCreate() to work with Jetpack Compose’s view setup and theming:

class MainActivity : ComponentActivity() {
    private val auth0Manager by lazy { Auth0Manager(this) }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        setContent {
            ChatGPTLoginTheme {
                // A surface container using the 'background' color from the theme
                Surface(
                    modifier = Modifier.fillMaxSize(),
                    color = MaterialTheme.colorScheme.background
                ) {
                    MyApp(auth0Manager = auth0Manager)
                }
            }
        }
    }
}


@Composable
fun MyApp(auth0Manager: Auth0Manager) {
    if (auth0Manager.userIsAuthenticated) {
        LogoutScreen(auth0Manager = auth0Manager)
    } else {
        LoginScreen(auth0Manager = auth0Manager)
    }
}

In the revised code, the MyApp composable uses auth0Manager’s userIsAuthenticated property — which Jetpack Compose observes, since it was declared using mutableStateOf() — to determine if it should display LoginScreen or LogoutScreen.

Login and Logout Screens

ChatGPT’s code for the LoginScreen view didn’t need any updating, even with my changes to Auth0Manager and MainActivity:

@Composable
fun LoginScreen(auth0Manager: Auth0Manager) {

    Column(
        modifier = Modifier.fillMaxSize(),
        horizontalAlignment = Alignment.CenterHorizontally,
        verticalArrangement = Arrangement.Center
    ) {
        Text(text = "Welcome to Your App", style = MaterialTheme.typography.headlineLarge)
        Button(onClick = {
            auth0Manager.login { result ->
                result.onSuccess { credentials ->
                    // Handle successful login
                    Log.d("LoginScreen", "Logged in")
                }
                result.onFailure { error ->
                    // Handle login error
                    Log.e("LoginScreen", "Login error")
                }
            }
        }) {
            Text(text = "Login with Auth0")
        }
    }
}

@Preview(showBackground = true)
@Composable
fun PreviewLoginScreen() {
    val auth0Manager = Auth0Manager(LocalContext.current)
    LoginScreen(auth0Manager = auth0Manager)
}

Here’s the resulting view, as seen in the Android emulator:

The Login screen. It has a headline, “Welcome to Your App”, and below it is a button that reads “Login with Auth0”.

Here’s ChatGPT’s code for LogoutScreen. Like LoginScreen, it didn’t require any updating on my part:

@Composable
fun LogoutScreen(auth0Manager: Auth0Manager) {
    Column(
        modifier = Modifier.fillMaxSize(),
        horizontalAlignment = Alignment.CenterHorizontally,
        verticalArrangement = Arrangement.Center
    ) {
        Text(text = "Logged In", style = MaterialTheme.typography.headlineLarge)
        Button(onClick = {
            auth0Manager.logout { result ->
                result.onSuccess { credentials ->
                    // Handle successful login
                    Log.d("LogoutScreen", "Logged out")
                }
                result.onFailure { error ->
                    // Handle logout error
                    Log.e("LogoutScreen", "Logout error")
                }
            }
            // Clear stored tokens and any other user data
        }) {
            Text(text = "Logout")
        }
    }
}

@Preview(showBackground = true)
@Composable
fun PreviewLogoutScreen() {
    val auth0Manager = Auth0Manager(LocalContext.current)
    LogoutScreen(auth0Manager = auth0Manager)
}

Here’s what the code produces:

The Logout screen. It has a headline, Logged In”, and below it is a button that reads “Logout”.

Conclusion

ChatGPT was able to write an iOS app that used Auth0 for login and logout, but it couldn’t do the same for Android. Here are my best guesses why this happened:

  • iOS’ SwiftUI framework has existed for two years longer than Jetpack Compose. Because of this, there’s more information and documentation for it than there is for Jetpack Compose, which makes it more likely that ChatGPT’s training data has more SwiftUI-related material than Jetpack Compose-related material.
  • Android programming is more complex. In addition to the fragmentation issue, Android features a more complex object model (for example, there’s a ViewModel class and an AndroidViewModel class, and each has its own benefits and drawbacks).
  • Auth0 mobile material: right now, there’s a little more material covering Auth0 integration for iOS than there is for Android. As the developer advocate directly in charge of mobile-related matters, I’m working on bringing these into balance.

ChatGPT and other large language models continue to evolve. I wouldn’t be surprised if it answered my question with working Android code six months from now.

Development in the age of AI

As AI developers continue on what appears to be a self-destructive mission of turning machine learning systems into tireless programmers who don’t need paychecks, you’re probably wondering what you should do to prepare.

In my earlier article on ChatGPT and iOS, I pointed to this IEEE Spectrum article How Coders Can Survive—and Thrive—in a ChatGPT World, which lists four ways in which developers can stay ahead of generative AIs.

You should also read this article on The New Stack: 7 Ways to Future Proof Your Developer Job in the Age of AI. Its thesis is that AI is already changing the way developers work, and we should adapt by harnessing it to do the things that computers excel at so that we can focus on those things where humans outperform computers.

  • Twitter icon
  • LinkedIn icon
  • Faceboook icon