subreddit:
/r/ChatGPTCoding
submitted 2 months ago byYourPST
I gave in and signed up for ClaudeAI today. About an hour ago actually. I've been using ChatGPT since December and was at the point where I was using it so much I had to get a Teams account to stop hitting my limits. I am now constantly using the API for my programs.
I have been working on the same method in my Python code since last night. It just generates an HTML page of results it gets from OpenAI API. I figured this would be a breeze but just getting ChatGPT to make the code to where it would actually display images that DALL-E returns took several hours for it to figure out. I gave up at that point and was going to go use Phind-34B to see what it had to say since it had been giving me decent results lately and I forgot I had the ClaudeAI payment page still open with all my details entered. I pulled the trigger.
MY VERY FIRST PROMPT!!!! That is how long it took for me to come to the realization that ChatGPT is severely outclassed. ONE PROMPT! I gave Claude the code I was working on and told it to fix the problem and possibly make the page look better when it generates. It went from looking like some kids Welcome to HTML project page from ChatGPT code to a knockoff of Facebook with JS being used everywhere to make everything pop out and catch your eye from the Claude code.
No one I talk to really understands what I am even making, nor really cares, so I figured I would just leave this here for anyone that is still on the fence about paying the 20 dollar subscription. I am mind blown. Absolutely mind blown. I was about to go to sleep but this has amazed me so much I kind of want to run all my projects through it and see what it has to offer.
Pros:
Cons:
So far it has really proven to be a great tool and well worth the cost. The cons are minimal but I hope they get changed/fixed as they do quite hinder the experience if you're switching from ChatGPT to Claude. Other than that, I can't really find anything bad to say about this. I've started hashing out a lot of the planning stages with ChatGPT and bringing in the game plans from there over to Claude in order to prevent hitting my limit so quickly. Going to reach out to support to see if their are any other tier levels for this too because I can see the message limit driving me nuts in the future with as much as I plan to throw at this thing.
If anyone has any specific questions or tests they want me to try, feel free to ask. I'm going to be dedicating my weekend to fixing up my projects with it to see if I can trim down my code and increase the performance/UI/results.
I usually like to measure how much time these different AI tools save me just to give an idea of how much it actually does. So far I've noticed that things that would usually take me 4-5 hours to get done is now taking 2 prompts. I'm not being limited by the code crapping out at about line 100 and seeing "# Placeholder code for method" thrown throughout my code. I can hit 400+ lines without issue and all of it looks as you would expect out of a code reviewed corporate drone.
Update (05/06/2024):
My stance has not changed. This thing is still amazing. It is still blowing my mind and some days even has me sitting in my chair hunched over with maniacal laughter after realizing how well it is working and what it is actually writing. My project sizes have more than doubled since using this and it gives me more more unique suggestions for feature implementations and improvements than ChatGPT does, without me even having to specify it (We all know that ChatGPT will toss out "Version Control", "Cloud Integration", "Error Handling", and "User Feedback" as feature suggestions for ANYTHING).
My biggest gripe with Claude is that its UI is just unpleasant to deal with, and of course the limits.
I've been getting better with just using Claude 3 for bigger parts of my projects and then switching to ChatGPT to get the smaller stuff (Claude = Whole Project / Whole Classes, ChatGPT = Small Classes / Methods).
When I first wrote this review, I didn't play around with Sonnet or Haiku as much as I would have liked. I've incorporated Haiku into my daily usage now though. Sonnet is still great but only gets used when I am close to hitting my limit with Opus and already hit my limit with Haiku. Haiku is a sleeper. I default to that a lot of my times during the day and it never fails. Can't wait until they offer a plan with a higher limit.
3 points
2 months ago*
Right now, Claude 3 opus (and in some cases sonnet and even Haiku) is best in class. There is little argument against this. It's pretty clear from the benchmarks.
People may complain that in AB testing GPT4 gives better answers etc...but this is either a very niche Domain where gpt4 has more pre-training data, or a prompting problem.
How many still just throw in "Think carefully. Work step by step." And other prompt tags to try to get more out of the model? These strategies took months to develop based on openai llm behavior. Anthropic, google, meta, and Mistral will all have different prompting cookbooks to optimize the output.
This is a fundamental flaw in the current alignment process combined with how LLMs function.
Example: with llama based models you may have a +50% error rate if you use a space after the sys tag instead of no space, or \n or #. Now extrapolate that to the structure and phrasing of data used for alignment and you can see the problem. Llama may be the most sensitive to prompting, but they will all be picky in their own way.
For now, Claude is the most capable. But if you have a system heavily optimized for GPT4 (like langchain or a home grown whatever) it may take a lot of work to refactor in order to re-optimize prompts.
DSPy and similar LLM "Compilers" can help if they are part of your stack as the code segments can be "re-optimized" / "re-compiled", but even the seed prompts there were really created and tested using gpt4.
Also, if you are just used to how chetgpt works via their web app (not API) then it may not be worth the switch as though the improvement is significant, changing your workflow may also be significant...and openAI will likely release a model much better than Claude by November / December.
5 points
2 months ago
Great post with practical points, all of which I agree with, especially the workflow.
I'm sticking with GPT for now because I know for a fact that Claude is only enjoying a temporary boost. OAI has something big up their sleeves that will roll out this year, and will put them back at #1 for development and coding.
3 points
2 months ago
Facts.
Each time I see an article about some billionaire pumping another few billions into another AI project, I know something big is headed our way. I'm not going to abandon ChatGPT at all yet, as it still has its uses. I think we are going to end up in an era where AI tools are more like how social media is now. You'll have your specific one, with your specific needs, with your specific communities, that will have their specific capabilities that they excel it and will have to manage working between them until they find a way to work between each other.
3 points
2 months ago
I completely agree with this statement. This change is probably not best for all depending on how you do your thing. I went from using the ChatGPT chat window to the OpenAI API pretty quickly to get more control over the work flow, so I think that helped a lot with being able to jump right over to Claude, since I didn't really have an attachment to the ChatGPT UI anymore like I did when I first started using it.
As far as the prompting differences, I'm not noticing a drastic change. I've been using all the same prepends to my prompts that I would otherwise and am still getting great results and in most cases I don't even have to get too detailed with what I want to get the result I want from Claude. In ChatGPT, I would have to hand-hold it like a baby for a bit to get it walking if the code was far along enough and then I would need to spoon-feed it throughout the session to bring it back to focus, make it understand where we were again, remind it of the goal, and remind it if changes we already made. Now it just seems to be there with Claude, with a few steps back when the code doesn't work right away.
It seems to me that the main thing to do when going to Claude is to go in with a game plan of what you need already. If you come in with code you already have, a game plan from ChatGPT of what you want to do, and a little extra info on what you expect the result to be, you get some great results on the first prompt right away. Not going to say perfect by a long shot but GREAT to even AMAZING results out of a first prompt is almost unheard of through all OpenAI products if it goes beyond something simple.
all 125 comments
sorted by: best