- cross-posted to:
- programmerhumor@lemmy.ml
- cross-posted to:
- programmerhumor@lemmy.ml
You must log in or # to comment.
The client wants to drag and drop their own personalized excel file with no guaranteed formatting or column order or data contract in order to import their data into our system <3
I love how this is a universal experience.
Needs more AI to randomly guess what the columns might be
Strangely enough we actually solved this problem with AI a few months back. We upload the excel file to Gemini and have a prompt to extract the data we need in a specific json format. And it works surprisingly well.
How well? Bet your life on it well, or “fewer hallucinations than we would have guessed” well? I’ve considered and toyed around with openAI models for logging supply room check offs in a JSON format and it went better than I hoped but worse than I needed.
Really well. Temp turned down all the way, and Gemini has this new feature to run and execute code… Not function calling… It can write a small python script, run it and return the output.
So our prompt explains the excel spreadsheet, then tell it exactly the format we need it in, and then tell it to use python and pandas to read in the CSV, clean it up and reshape it the way we need it to match what we expect and voila.
So hallucinations are not really and issue with the data as it’s simply writing code which then deterministically processes and returns the data.
Edit to add more info: basically Gemini can create and run a lambda function on the fly. And if you’re a coder you can really guide the prompt. Eg "load this into pandas. Then remove all the empty columns. Also remove the total rows. Now unpivot the data so the months are not columns but in separate rows with a column called month.
You get the idea.
Holy smokes, how did I not think to turn the temperature down?! That’s smart! Thanks for getting back to me!
Jesus, this gave me war flashbacks.
Please, do elaborate. Let others feast on your suffering.
May I?
A controlling department wasn’t granted any money for digitializing their workflow.
So these guys created their own solution(s!). Things like dedicated “user interfaces” loading data from tables created by hand. After years these people realized that data formatting is quite the issue.
They started to put random rules into different tables:
Two empty lines: New Group Data Record. One empty line: New Subgroup Data Record.
Excel tables aggregating this data via hardcoded links.
A dedicated table to start calculations on parent tables.
They mutated data like this:
Load data from excel files into one. Manually delete, add or change lines (or columns). Start a collection run from dedicated excel file and load new excel file data and replace old excel file data.
They had files where ‘it was easier to read’ when they pivot the data. This was troublesome since some values are intermediate results. Dropping one column may imply dropping another one as well.
All workflows required manual alignments along the way.
They were only able to process 10% of the data from a year within a year. Managing millions in cash.
Their data input came from different internal sources. Programs which were written two decades ago once and without any tests. Talking like VB, macro’s from host servers and copy-pasta data from other internal programs.
And don’t get me started on customer tables… They created a zip-code encoded filesystem hierarchy where each customer data (you guessed it, excel file) was renamed and then saved. In each of these directories where randomly named files if something went wrong; So no actual file patterns to rely on.
I respect them.
They creates a diagram for their tables with word. Word! (Didn’t know either: you can select the web view in the bottom right corner and you get an infitive canvas…) Madness.
I had a potential client, an accountant. They had their own, uh, system within a spreadsheet. They wanted me to program another system to be able to send their spreadsheet output into our governments IRS. Did a little back-and-forth but could not convince them to drop the idea.
Do we have the same client?
Everyone has and is that client.
If those project managers could read, they would actually be able to use Jira.
I regret what I wrote.
I realised, just being able to read, doesn’t make sure they can actually use Jira
I’ve been a professional software developer for over two decades. There is zero chance my job will get taken by an AI any time soon. Anyone who thinks my job is to write code doesn’t understand my job. That’s like saying a bus driver’s job is to turn a steering wheel.
My job is to turn vague ideas and nondescript feelings into APIs and (sometimes) UIs, then turn those into specs, then split those into tasks, then sometimes I’ll write the code for them and sometimes someone else does. About 90% of my time is turning ideas into plans, and about 10% of my time is turning those plans into code.
When I was young and was a junior engineer, my job was more to receive the specs from the senior engineers and turn that into code, but even then, I was still designing my own stuff. Maybe more like 40/60 time instead of 90/10.
Now that I’m a grizzled old man forged in the fires of task management software, I’m doing almost all of the design work myself. I manage a project that has about 250,000 lines of code. An AI isn’t going to be able to build new features into that, let alone decide which features to build in the first place.
It won’t automate your job, but isn’t it already replacing junior devs via team downsizing?
Any place that is replacing junior devs with AI is probably going to really regret it when they have no senior devs in a few years. Being a junior dev in a team is kind of like an apprenticeship. You learn the trade, but you also learn the shop. Then when the senior dev moves on, you have all that knowledge and can step into the role of senior dev. If a team decides to not have junior devs anymore, then they’ll have no one to take over when a senior dev leaves.
So the answer is yes, it is already replacing junior devs, but that’s only because management hasn’t learned how bad of an idea that is yet. Ultimately, it will cost them more through losing foundational team knowledge.
You also have to hold an AI’s hand the entire way through coding something, whereas you can kind of just let a junior dev go do their own thing, and eventually they’ll probably get it right. An AI “agent” tries to hold its own hand, but that doesn’t seem to work out usually when I’ve tried it. It starts making changes that are really bad, then just seems to always double down and eventually make a huge mess.
So I’m not wasting my time in college then?
Last company I worked for and now contract for, explicitly set out to hire promising juniors over seniors. Reason being, they had to fire a guy with nearly a decade of experience because he was completely unable to adapt and learn new things, so his experience was all doing the same stuff over and over again.
A small company that has cash reserves will absolutely hire a bright grad who can hold a conversation in the interview, only trouble is the ratio between candidates and job openings.
Heh, that won’t stop a C-level from thinking that you just write code.
Yeah, that’s probably true. Remember how all the execs decided to replace cashiers with robots, then the stores started losing money because a. it made stealing a lot easier and b. people would avoid stores that only had self-checkout robots and never had anyone to help you because a robot doesn’t know where the flour is. Now the self checkouts are being decommissioned and we’re going back to regular human cashiers. It turns out cashiers do more than just scan barcodes. But, upper management didn’t get to where they are by being smart.
Now the self checkouts are being decommissioned and we’re going back to regular human cashiers.
Maybe this is North American thing because in Europe they never really got rid of human cashiers, they just had the automated systems alongside the human cashiers.
I don’t know of any store that went over to 100% self-checkout
No, the customer wants a button that does a very specific thing.
He can’t tell you what that is, though. You’re the expert!
Also, can you put in more ads? And make it so the users can’t close the tab until they bought something.You’re the expert!
I can do absolutely anything. I’m an expert!
