Pages

Tuesday, 9 May 2023

ChatGPT Meets de Bono: A Powerful Combo

I'm struggling to make a decision at the moment, so I've enlisted the help of ChatGPT.

I am experimenting a lot with some of the new AI tools, and it's a bit like playing whack-a-mole at the moment; as soon as I get to grips with one bit of software, another appears and demands my attention.

A lot of what I'm doing is based on OpenAI's ChatGPT. I'm concerned that I'll spend too much on paid services, so I am actively looking at Open Source alternatives that I could run locally.

Jetson Xavier dev kit
Alas, many Large Language Models (LLMs) are too resource-hungry to run on my workstation, and I've been thinking of trying out a Jetson Xavier which I'm not using at the moment.

Worth the time?

I know I'll need to spend a few days getting to the point where I know whether I can use it for the projects I have in mind. Should I invest the time?

I decide to take a Judo approach and use ChatGPT to help me make the decision.

I used a prompt that's worked well for me before now. It based on de Bono's Six Thinking Hats.

You've probably come across the Six Thinking Hats technique before, but if not you can read about them on Wikipedia.

The Prompt and Response

Here's my question, along with ChatGPT's response. I've made minor tweaks to the format but the text is unchanged.

I'm thinking of running an LLM as an alternative to ChatGPT on my 16 GB Jetson Xavier developer kit.
Use Edward de Bono's Six Thinking Hats to help me evaluate that idea.

ChatGPT

Edward de Bono's Six Thinking Hats is a powerful decision-making tool that encourages looking at a problem or idea from
multiple perspectives. In this case, we'll use it to evaluate running an LLM (Large Language Model) as an alternative to
ChatGPT on a 16 GB Jetson Xavier developer kit.

White Hat (Facts and Information):

  1. The Jetson Xavier developer kit has 16 GB of RAM and a powerful GPU, making it suitable for running AI models.
  2. LLMs typically require significant computational power and memory.
  3. ChatGPT is based on GPT-4, which is also an LLM.

Red Hat (Emotions and Intuition):

  1. You might feel excited about running an LLM on your Jetson Xavier kit, as it could be a challenge and an opportunity to learn.
  2. Concerns may arise about whether the hardware is capable of handling such a large model efficiently.

Black Hat (Critical Judgement):

  1. Running an LLM on a Jetson Xavier developer kit might be limited by the 16 GB of RAM, depending on the size of the model.
  2. Performance and latency might be issues due to the large size of the LLM, affecting real-time applications.
  3. Power consumption and heat generation could be concerns when running a large model continuously.

Yellow Hat (Optimism and Benefits):

  1. Implementing an LLM on the Jetson Xavier kit could lead to a deeper understanding of AI models and their deployment on edge devices.
  2. It could potentially offer better performance compared to smaller models, resulting in more accurate and contextually aware responses.
  3. Developing and optimising an LLM for the Jetson Xavier kit could contribute to the growing AI community.

Green Hat (Creativity and Alternatives):

  1. Consider using a smaller, more efficient model that is optimised for edge devices, while still providing a satisfactory level of performance.
  2. Explore the possibility of using model distillation or pruning techniques to create a smaller model suitable for the Jetson Xavier kit.
  3. Investigate other hardware solutions or cloud-based platforms that may better accommodate the computational requirements of an LLM.

Blue Hat (Process and Summary):

In summary, running an LLM as an alternative to ChatGPT on a 16 GB Jetson Xavier developer kit has its pros and cons.
While it may offer better performance and accuracy, it might also present challenges related to hardware limitations,
power consumption, and heat generation. Consider exploring alternative models or hardware solutions, or optimizing the
LLM for edge devices. Ultimately, the decision will depend on the specific use case and the desired balance between
performance and efficiency.

Not bad for an AI!

I'm impressed by the response, as I have been with other uses of the prompt. I've also used two other de Bono frameworks: Six Value Medals, and Six Action Shoes. You can ask ChatGPT about them.

Pro tip: Make sure you are using ChatGPT 4, as version 3.5 has confused Action Shoes with the Thinking Hats!

The books are less well known but they can form the basis of useful prompts.

You'll also see from the Wikipedia article about Six Thinking Hats that there are recommended sequences of Hat use for tackling particular types of problem. These could form the basis of further useful prompts for ChatGPT.

I'll share more useful prompts as I discover them. You can keep track of them by following me on Twitter at @rareblog.


Monday, 8 May 2023

Finding the Best Solution with ChatGPT: A Personal Experience

When you're developing software for technically savvy users, you'll often encounter an annoying problem: users provide their preferred solution instead of their requirements.

This issue is even more pronounced when you are both developer and user.

In such situations, you're likely to choose the first solution that seems to meet your needs, even though there might be better alternatives.

Pair programming can help, as your partner can spot what's going on and suggest other ideas. But what if you're programming on your own?

That's where ChatGPT comes in handy.

ChatGPT to the Rescue

I've recently been working on an application that downloads interesting arXiv PDFs, and I wanted to access them remotely. I could think of a couple of ways to do that, but I decided to ask ChatGPT for advice. I used a prompt that had been helpful before:

"Suggest three ways I could accomplish that, with the pros, cons, and interesting features of each approach."

I presented my problem: I have a directory containing PDFs on a Raspberry Pi computer, and I want to be able to read the PDF files from Python on another Linux computer.

ChatGPT suggested three approaches, complete with pros and cons for each:

  1. File sharing with Samba
  2. Remote access with SSHFS
  3. Web server with a file browser

I had considered using Samba shares or a web server, but I had forgotten about SSHFS. Since I already had password-less SSH access to the Raspberry Pi, SSHFS was incredibly easy to set up, and it has been working very well.

Reusable Prompt for Problem-Solving

Here's the reusable prompt I mentioned earlier:

"Suggest three ways I could accomplish that, with the pros, cons, and interesting features of each approach."

I now use this prompt whenever I'm trying to find a solution to a problem, and ChatGPT consistently delivers valuable insights.

I'll share more useful prompts as I discover them. You can keep track of them by following me on Twitter at @rareblog.

Friday, 28 April 2023

Pair Programming with ChatGPT

 

Earlier today I started work on a small admin project to help me organise my excessive collection of repositories on GitHub.

I wanted a list of my repositories, showing the name, url, description, stars and forks.

I decided to get ChatGPT to write the code for me.

In the conversation that follows, you can imagine an embodied ChatBot sitting next to me as we pair program. I'm the customer, ChatGPT is the programmer. If I'm not happy with the results, I ask my pair to change the code.

My part is in italics. Everything else came from ChatGPT.

As you'll see, there are a couple of wrinkles I didn't anticipate, but ChatGPT fixed them.

This took less time than I'd have taken to write the code, and it was much more fun. I recommend it.

Here's the session.

Write python to find the names of all my repositories on GitHub, My username is romilly.

You can use the GitHub API to retrieve the names of all your repositories. Here is the Python code to achieve that:

import requests

# Replace <username> with your GitHub username
username = "romilly"

# Make a GET request to the GitHub API to retrieve your repositories
response = requests.get(f"https://api.github.com/users/{username}/repos")

# Check if the response was successful
if response.status_code == 200:
    # Loop through the list of repositories and print their names
    for repo in response.json():
        print(repo["name"])
else:
    print("Failed to retrieve repositories")

This code uses the requests library to make a GET request to the GitHub API endpoint that retrieves your repositories. It then loops through the list of repositories in the response JSON and prints their names.

Modify that to output the name, url, description, stars and forks

Sure! Here's the modified code that retrieves and prints the name, URL, description, stars, and forks count of all your repositories:

import requests

# Replace <username> with your GitHub username
username = "romilly"

# Make a GET request to the GitHub API to retrieve your repositories
response = requests.get(f"https://api.github.com/users/{username}/repos")

# Check if the response was successful
if response.status_code == 200:
    # Loop through the list of repositories and print their details
    for repo in response.json():
        print(f"Name: {repo['name']}")
        print(f"URL: {repo['html_url']}")
        print(f"Description: {repo['description'] or 'No description available'}")
        print(f"Stars: {repo['stargazers_count']}")
        print(f"Forks: {repo['forks_count']}")
        print("="*50)
else:
    print("Failed to retrieve repositories")

This code uses the same requests library to make a GET request to the GitHub API endpoint that retrieves your repositories. It then loops through the list of repositories in the response JSON and prints their name, URL, description, stars count, and forks count. If the repository has no description, the code prints “No description available” instead. Finally, the code prints a line of equal signs (=) to separate each repository's details.

That only prints the fist few repositories. Modify the code to paginate though the full set.

You're right! By default, the GitHub API only returns the first 30 repositories, so we need to paginate through the full set to retrieve all your repositories. Here's the modified code that uses pagination:

import requests

# Replace <username> with your GitHub username
username = "romilly"

# Set the base URL for the GitHub API endpoint that retrieves your repositories
base_url = f"https://api.github.com/users/{username}/repos"

# Initialize the page number to 1
page = 1

# Loop through the pages of repositories
while True:
    # Make a GET request to the current page of the GitHub API endpoint
    response = requests.get(f"{base_url}?per_page=100&page={page}")

    # Check if the response was successful
    if response.status_code == 200:
        # If the response is empty, we have reached the last page of repositories
        if len(response.json()) == 0:
            break

        # Loop through the list of repositories on the current page and print their details
        for repo in response.json():
            print(f"Name: {repo['name']}")
            print(f"URL: {repo['html_url']}")
            print(f"Description: {repo['description'] or 'No description available'}")
            print(f"Stars: {repo['stargazers_count']}")
            print(f"Forks: {repo['forks_count']}")
            print("="*50)

        # Move to the next page
        page += 1
    else:
        print(f"Failed to retrieve repositories from page {page}")
        break

This code initialises the page number to 1 and loops through the pages of repositories using a while loop. It makes a GET request to the current page of the GitHub API endpoint with a per_page parameter of 100, which is the maximum number of repositories that the API allows to be returned per page. If the response is successful and not empty, the code loops through the list of repositories on the current page and prints their details. Finally, the code moves to the next page and repeats the process until there are no more repositories to retrieve. If the response is unsuccessful, the code prints an error message and breaks out of the loop.

What's the rate limit for GitHub API requests?

The rate limit for unauthenticated requests to the GitHub API is 60 requests per hour, while the rate limit for authenticated requests varies based on your authentication status and whether you are using a personal access token.

Here are the rate limits for authenticated requests:

Personal access token: 5,000 requests per hour
OAuth token: 5,000 requests per hour
OAuth token with public_repo or repo scope: 5,000 requests per hour
GitHub App installation token: 5,000 requests per hour
GitHub Actions token: 1,000 requests per hour

It's important to note that the rate limit is calculated on a per-IP address basis, so if multiple people are using the same IP address to make API requests, their combined rate limit will be shared.

To check your rate limit status, you can make a GET request to the following endpoint:

https://api.github.com/rate_limit

This endpoint returns a JSON object that contains information about your current rate limit status, including the number of remaining requests and the time at which the limit resets.

Modify the code to check the rate limit before each request. If the limit has been reached, delay the request until the limit resets.

Good idea! Here's the modified code that checks the rate limit before each request and delays the request until the limit resets if it has been reached:

import requests
import time

# Replace <username> with your GitHub username
username = "romilly"

# Set the base URL for the GitHub API endpoint that retrieves your repositories
base_url = f"https://api.github.com/users/{username}/repos"

# Initialize the page number to 1
page = 1

while True:
    # Check the rate limit status before making the request
    response = requests.get("https://api.github.com/rate_limit")
    if response.status_code != 200:
        print("Failed to retrieve rate limit status")
        break

    remaining_requests = response.json()["resources"]["core"]["remaining"]
    reset_timestamp = response.json()["resources"]["core"]["reset"]

    if remaining_requests == 0:
        # If we have reached the rate limit, delay the request until the limit resets
        delay_seconds = reset_timestamp - time.time() + 1
        print(f"Rate limit reached. Waiting for {delay_seconds:.2f} seconds...")
        time.sleep(delay_seconds)

    # Make a GET request to the current page of the GitHub API endpoint
    response = requests.get(f"{base_url}?per_page=100&page={page}")

    # Check if the response was successful
    if response.status_code == 200:
        # If the response is empty, we have reached the last page of repositories
        if len(response.json()) == 0:
            break

        # Loop through the list of repositories on the current page and print their details
        for repo in response.json():
            print(f"Name: {repo['name']}")
            print(f"URL: {repo['html_url']}")
            print(f"Description: {repo['description'] or 'No description available'}")
            print(f"Stars: {repo['stargazers_count']}")
            print(f"Forks: {repo['forks_count']}")
            print("="*50)

        # Move to the next page
        page += 1
    else:
        print(f"Failed to retrieve repositories from page {page}")
        break

This code first makes a GET request to the rate limit endpoint to retrieve the number of remaining requests and the time at which the limit resets. If we have reached the rate limit, the code calculates the number of seconds until the limit resets and waits for that amount of time using the time.sleep() function. After the delay, the code makes the GET request to the current page of the GitHub API endpoint as before. If the response is successful, the code loops through the list of repositories on the current page and prints their details. Finally, the code moves to the next page and repeats the process until there are no more repositories to retrieve. If the response is unsuccessful, the code prints an error message and breaks out of the loop.

What next?

I'm pretty happy with what ChatGPT did, but just like a real user I still want some changes - a database and a GUI to query it. I'll see if ChatGPT has time to do that and let you know.

I'll be posting on twitter as RAREblog.

If you found this interesting, please let others know: