With so much talk about ChatGPT replacing jobs (which is in fact a true concern). I wondered if ChatGPT could do my job for a day. (Mostly because it was the weekend and it’s more challenging to work on the weekend.) I asked it to do a couple of tasks. Here are some results.
Task 1: Create a test that tests job skills
I asked ChatGPT to create some questions to test a job candidate’s job skills. It generated an answer in a matter of seconds:
Sure! Here are some sample questions you could use to test a job candidate’s proficiency at customer service:
- How do you handle a customer who is upset or dissatisfied with the product or service they received?
- Can you provide an example of a time when you went above and beyond to help a customer?
- How would you deal with a customer who is speaking loudly or rudely to you?
- Describe a time when you had to handle multiple customer requests simultaneously.
- How do you stay calm and composed when dealing with difficult or angry customers?
- What do you think are the most important qualities for someone in a customer service role?
- How do you handle a situation where you are not able to provide a customer with the solution they were hoping for?
- Have you ever dealt with a language barrier when speaking with a customer? How did you handle it?
- How do you keep yourself organized when handling multiple customer requests or issues?
- Can you provide an example of a time when you successfully turned a dissatisfied customer into a loyal one?
Task 2: Create an answer key that can be reviewed using AI
Perhaps I didn’t ask the right question. When I asked ChatGPT to create an answer key to a job skills test using AI, it came up with this response:
To have AI check the answers to these questions in a standardized way, you can follow these steps:
- Develop a set of criteria for evaluating each question. This could include factors such as the clarity and completeness of the answer, the use of relevant examples, and the candidate’s ability to communicate effectively.
- Train a natural language processing (NLP) model on a large dataset of customer service responses to the questions you have developed. This will enable the model to recognize and understand common patterns and language used in customer service responses.
- Use the trained NLP model to automatically evaluate the candidate’s responses to the questions. The model will analyze the text of each response and score it according to the pre-defined evaluation criteria.
- Set a threshold for acceptable scores for each question. Responses that score below this threshold can be flagged for review by a human evaluator.
- Review the results of the automated evaluation and make any necessary adjustments to the evaluation criteria or the threshold scores.
Conclusion:
When I asked ChatGPT to create a job skills assessment as well as a method to evaluate the assessment using AI, it replied with well-written responses. However, none of it was material I could use because the material was far too general. The issue was also my inability to word my task to get the answer I really wanted: to get ChatGPT to do my homework. What I learned was that it may take some skill to learn how to get ChatGPT to read my mind and do my creative thinking so I can take an afternoon off.
If you enjoyed this article, subscribe to make sure you don’t miss the next post!
Great ideas! Well shared
LikeLiked by 1 person
Thanks! Have a great day.
LikeLiked by 1 person
Visit my blog
LikeLiked by 1 person
Interesting idea!
LikeLiked by 1 person
Thanks!
LikeLike
It makes sense that it would be general. Best way to play it safe.
LikeLiked by 1 person
Agreed.
LikeLiked by 1 person
I think that ChatGPT is so great for many applications, but I still have mixed feelings about using it. I won’t be using it for my blog posts or for writing my novels. Thanks for sharing your interesting experiment.
LikeLiked by 1 person
I’ve experiemented with it a few times for fun and a few times because my client asked me to. What I’ve found so far is it lacks the sophistication needed for test design. That’s good news for me – my job is still secure. 🙂
LikeLiked by 1 person
Really interesting, I would be interested in seeing whether it could answer better in the future.
LikeLiked by 1 person
I would be interested as well, but there are some challenges to this. For example, some writers have NDAs preventing them from using ChatGPT because anything ChatGPT generates became a part of its knowledge base. So if I ask it to create test questions, those questions cannot be confidentially used by a client.
LikeLiked by 1 person
Agreed. Seems like many are experimenting with it.
LikeLiked by 1 person
Funnily enough, my bosses asked me to experiment with it.
LikeLiked by 1 person
Nice.
LikeLiked by 1 person
Worth trying again in 6 months to see how quickly it adapts!?
LikeLiked by 1 person
I’ll be following the progress of anything similar to ChatGPT, not just ChatGPT because my work involves developing an AI product.
LikeLiked by 1 person
Oh Brave New World …. 😉
LikeLiked by 1 person
Yes. History has yet to be written….
LikeLiked by 1 person
I use Chat GPT for some research purpose, and nothing much.
I worked with it only once. But the answers are somethimes not satisfactory.
LikeLiked by 1 person
Yes, I think it is good for research, but not there yet for getting the exact information we need.
LikeLike
True!!
I used it for 1 specific blog, it gave me some information.
But overall reading articles and getting info from other sources is much better
LikeLiked by 1 person
This is true. The information I get is far too general. I find it easier to do my own research.
LikeLiked by 1 person
Interesting thing to try.
LikeLiked by 2 people
It was worth a shot. ChatGPT has been talked about so much lately.
LikeLiked by 1 person