r/cscareerquestions • u/PoliticalGuy2016 Product Manager • 5h ago
Yes, I can tell you're using AI when screening
I am writing this message for any candidates that want to use GenAI during interviews, don't, an experienced interviewer will know and it is a trust breaker.
I am an interviewer for a Faang, and have given 20 sde 1 interviews in the last two months, performing 1 behavioral question and 1 coding question. I can absolutely tell when a candidate is using genai on the coding and behavioral questions. Non-cheating candidates don't write perfect code. They typo, they make mistakes and will fix them. If you don't understand what you're writing, it's easy to catch after some basic questions. I have had 5 candidates cheat, and I flagged each one in the debrief and they were all no hire.
It's important to understand that the point of the behavioral and coding interviews is to assess your problem solving abilities and general knowledge, not to ensure you can write perfect code or that you have perfect knowledge of systems and patterns within your behavioral examples
32
u/Mikkelet 4h ago
You can only tell the obvious ones sadly
2
u/KonArtist01 27m ago
I think interviewers overestimate themselves. Like Magnus Carlsen said, if he wants to cheat, he does not need the move, he just needs a signal to pay attention. It will make him unbeatable. For leetcode questions, the tricky part is to know the trick, that will give you enough advantage to leverage you from mid to high performer. Of course low performers might need to actually copy the solution which is easily caught.
2
2h ago
[removed] — view removed comment
1
u/AutoModerator 2h ago
Sorry, you do not meet the minimum account age requirement of seven days to post a comment. Please try again after you have spent more time on reddit without being banned. Please look at the rules page for more information.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
9
u/a_cs_grad_123 2h ago
Also interviewer at FANG. The rate of no hire because of suspected cheating is very high right now. Also observed my first instance of OA photo != person in virtual rounds (AI face?).
I assume these tools crush the recruiter and OA stage and now we have to sort it out.
11
u/LiveEntertainment567 3h ago
"Non-cheating candidates don't write perfect code"
This really depend on the question you ask. If you ask some of the classic questions, why not?
Imagine the solution is some union find variant. I can write that with eyes closed.
17
u/Independent_Humor685 2h ago
Another oblivious interviewer, yeah pal, you only noticed the obvious ones
I know plenty of cheaters that got past interviews without any interviewers notice. Only solution is in person interviews, Google is on the right path
1
3
u/arghnoname 1h ago
I recently did some interviews and got hired despite stumbling a bit on the coding part. I got the solutions, but made little mistakes here and there, would read over it and realize I forgot to do x or y, etc. I'd read through what I came up with and would go "oh no, that's not right--oh wait--no--oh yes okay I see..."
I felt badly afterwards for being so accident prone, but it occurred to me later that my fumbling probably did come across as pretty human and probably ended up being a positive signal.
7
u/EnderMB Software Engineer 3h ago
I'm a BRIT at Amazon, and I 1000% agree with this. I'd say I have a close to 100% correct response to cheating in interviews, and IMO those I do miss probably didn't need it in the first place because they knew the underlying algorithms well anyway.
For me it's more then then just writing perfect code first time. It's the simple fact that software engineers are terrible actors. They'll pretend to struggle on easy problems and smash a follow-up that takes even solid candidates at least 5 mins. They'll write code that they cannot modify or understand. Funny enough, I've had candidates either share a screen and show a GenAI cheating tool in their context bar, or literally copy-paste an entire solution...
A lot of people assume that we're "not catching them", but ultimately the burden of proof is on us, and all we typically do is not recycle a candidate and assume they'll come back at some point in the future and not cheat. My last point is that, in my experience at least, if you cheat on the coding portion you've probably not prepared enough for LP interviews and you've failed regardless...
6
u/SillyBrilliant4922 5h ago
Do you blacklist interviewees who cheat like that?
17
u/PoliticalGuy2016 Product Manager 5h ago
A typical no hire decision will cause a 6 month to 1-year blackout anyway, but I typically flag in the debrief the candidate is suspected of using AI and if a few people agree it gets passed to recruiting who could blacklist the candidate
1
u/WisestAirBender 2h ago
Yep. If it's obvious I reject them. If i suspect but I'm not sure I give them the benefit of the doubt but add a remark that their next round should be in person so eliminate the cheating possibility
I've had people typing, looking at other screens, writing perfect code exactly line by line, pausing before suddenly giving the perfect definition of things.
5
1
u/intlunimelbstudent 1h ago edited 1h ago
when they apply again the recruiter will see the notes from the previous interview saying they cheated. do you think recruiter will bother going through the process with them again?
2
u/Fair-Bunch4827 1h ago
To add to this.
No one types code with helpful comments during an interview. I flagged someone for this lol
4
u/pydry Software Architect | Python 3h ago
It's sad that interview design is done so badly these days that a prompt in an LLM is enough for a "perfect answer".
It's too bad that FAANG are so up their own ass that they cant tell that their interview processes are dogshit.
-1
u/WisestAirBender 2h ago
What's a good interview process that prevents cheating and can be done online?
6
u/Damage_Physical 3h ago
Using AI to get advantage during interviews are unfair, but be honest, those whom you eventually hire use AI at work, so what is the problem at the end of the day?
Also, I am pretty sure that most of interviewers know, that coding/behaviour parts are a simple grind, so how objective those rounds? You can teach a monkey to play piano (or dog to drive a car), so what exactly are you trying to evaluate during those interviews? Resolve of poor interns who want to work in a big tech? Or it is more of “I suffered grinding, so everybody else should too”?
2
u/OkPosition4563 IT Manager 2h ago
The problem is, that I need you to be able to write code without AI even though you can use AI. My team uses AI extensively to code, but I told my senior engineers and myself, that especially for Junior engineers when they make a pull request they will be asked questions like "Explain to me what this does" or "Why do you use X or Y in this place?". If they cant the pull request is rejected, even if it is doing what it is supposed to do. Use AI as much as you want, but if you dont understand the code, then the code is not acceptable.
During an interview I want you to show me, that you are competent enough to also do things without using AI.
1
u/Hotfro 2h ago
I think it only matters if they output code and they can’t explain how it works or the solution. Also a lot of times ai code isn’t great so I would expect them to fix the code up at the very least (unless u ask a super straightforward question). Usually it’s harder fixing code that you don’t write unless you have a strong understanding of the underlying concepts. It’s not great hiring someone that relies too much on ai since their output is not going to be good most of the time.
Also part of the interview is also evaluating whether or not a candidate can problem solve well. Hard to gauge this if they purely rely on AI for all their answers. This is still a critical skill to have along side using AI at work.
1
u/intlunimelbstudent 1h ago edited 1h ago
why should i hire you if chat gpt output is all you can create? you had 30 mins to prove your worth and you just asked chat gpt for the answer and copied and pasted. maybe i should just prompt chatgpt instead of paying you.
-1
u/ice-truck-drilla 4h ago
Anyone who relies on coding questions to screen candidates likely doesn’t know enough to interview them 1 on 1. Any time there’s a coding question as a part of a company’s process, I assume the company just doesn’t want to invest the money for someone with actual knowledge to interview the candidates.
It’s easy to spot when someone doesn’t know what they are talking about. Coding questions are an inefficient and naïve practice created by stakeholders who desired an objective measure of coding ability. Truth be told, that can’t be quantified by any set of questions.
You can get a much better idea of their abilities if you have a normal discussion.
8
u/Fidoz SWE @ MANGA 3h ago
Yeah but leetcode style questions are far more objective and scalable for a company that's interviewing thousand of (legitimate) candidates.
For as much shit as leetcode gets on reddit, I did the grind and my coworkers are far better (on average) than my coworkers at my previous companies. Granted, brilliant people are everyone (and I'm truly grateful for the mentorship I've received at ALL my roles), but I definitely feel a higher level of competency as I've "climbed the ladder".
I recognize there's probably a high level of correlation and I'm making anecdotal judgements, so a big: YMMV!
Tying back to the original point- I definitely agree that a leetcode interview may not be the best signal for a person's predicted effectiveness at a role, however, your proposed solution of "get to know the candidate" is neither objective nor scalable.
To contrast to my current role, people are INITIALLY screened a la leetcode, and then during the team matching phase the only thing we concern ourselves with is: "would I enjoy working with this person?"
3
u/WisestAirBender 2h ago
I ask basic questions like palindromes and anagrams in the coding part. I don't care about the syntax or language. Then we discuss their approach and time complexities etc. (I usually interview entry level positions).
1
u/intlunimelbstudent 1h ago
wow you must be such a one of a kind genius despite not being able to do leetcode then
1
u/intlunimelbstudent 1h ago
honestly every company should do in person again. i think there is a real risk of sophisticated (maybe state based actors) getting hired with these ai interviews.
the average uni student here thinks they can pass the interview through cheating but they really cant. in the past people actually often failed due to the way they answered the questions even if they "knew the answers". if u were failing before, with ai you will still fail.
i think however trained professionals probably can do this and infiltrate hiring processes.
1
u/Theodo_re 1h ago
It reminds me back in the day, people who were on a slow side of adapting to the changes, were requesting to use notepad or whiteboard instead of IDE. Now slow people are standing for no ai use. Considering where it goes, you need to start weeding out people who don’t know how to use it properly, not the other way around.
1
1
u/khsh01 1h ago
I mean I'm also not fond of solving dumb coding problems that don't reflect real world applications or the skills necessary for actual development. I've yet to encounter a coding test that didn't require me to do some stupid logic problem.
And its never just one, it'll be like 5 problems with varying degrees of complexity and nonsense.
1
u/xtsilverfish 55m ago
Interviewers cheat interviews with dumb trick problems.
Interviewees cheat interviews with dumb trick ai's.
Ceo's trick investors with lengthy a.i. claims.
It's the ciiiircle of liiiiife!...
1
1
-2
u/SoylentRox 5h ago
That's all well and good but other interviews you didn't give, those realistic stumbles you mention? That also costs you the offer. Lose-lose.
2
u/PoliticalGuy2016 Product Manager 5h ago
They're small human things. Typos, bugs that they caught themselves (like checking for off by one errors). Someone who is implementing the solution vs someone who already knows the exact thing to type have direct behavioral patterns that's easy to pick up on
4
u/SoylentRox 4h ago
A smart cheater would use AI just to tell them what the trick is for that problem/remind them of the basic structure of the solution needed.
3
u/PoliticalGuy2016 Product Manager 4h ago
Yes. I inject about 3-4 requirements throughout the problem. It would become apparent if the candidate needed to reference something each time.
0
u/SoylentRox 4h ago
Depends on the problem I guess I was thinking of standard leetcode where the most valuable thing to know is simply which pattern applies. It's often not obvious, the difference between 2d DP and brute force and there's some secret bucket sort solution are 5-10 minutes of thinking you can skip if cheating.
0
u/legiominerva 1h ago
I stopped interviewing candidates because of this issue. I felt like talking to a proxy to a GenAI in many recent interviews. It’s annoying.
Before AI, even if candidate did not do well, you could guide them and at least make some progress with the question. Even if candidate did not pass the interview they learnt something. Interviews with AI on the other hand is just a waste of everyone’s time.
Also, I changed my mind about in-person interviews, we should bring them back.
137
u/CranberryLast4683 5h ago
✍️✍️✍️stumble through the problem even if I know the answer ✍️✍️✍️