Boris Steipe @boris_steipe@qoto.org

I just posted a significant bit of analysis for Sentient Syllabus, a proof of concept for personalized assignments.

https://sentientsyllabus.substack.com/p/assignment-and-agency

In the eighties, Benjamin Bloom (the Bloom of Blooms taxonomy) reported that one on one instruction can boost student performance by two sigma! Ever since then, we have been searching for ways to scale this for our current realities in #education - but no breakthrough has appeared. Why are we not teaching this way if the results are so compelling? Because, as Bloom said: "it is too costly for most societies to bear on a large scale".

With the arrival of generative AI that limit will change.

I have worked out a proof of concept for personalized assignment design that needs only a spreadsheet and #ChatGPT. The spreadsheet builds a prompt that students can customize, ChatGPT writes out the assignment. No specialized software, technology, or third party involvement are needed.

Of course, the results need to be vetted - but the improvement becomes part of the learning process, and overall the process hands agency for learning back to the student: the assignment becomes theirs.

The proof of concept is done from the perspective of a Computational Biology course I taught last year - but adapting it to other fields of higher education should be trivial. There is nothing inherently STEM like in the approach - humanities, writing, language learning ... no reason it would not work for secondary education as well.

The potential is remarkable.

I encourage you to boost and share - this will be valuable for educators at all levels, and it will give us very concrete ways to harness the new opportunities.

:-)

I just posted a significant bit of analysis for #SentientSyllabus, a proof of concept for personalized assignments.

https://sentientsyllabus.substack.com/p/assignment-and-agency

In the eighties, Benjamin Bloom (the Bloom of #Bloomstaxonomy) reported that one on one instruction can boost #studentperformance by two sigma! Ever since then, we have been searching for ways to scale this for our current realities in #education - but no breakthrough has appeared. Why are we not teaching this way if the results are so compelling? Because, as Bloom said: "it is too costly for most societies to bear on a large scale".

With the arrival of #generativeAI that limit will change.

I have worked out a proof of concept for personalized #assignment design that needs only a spreadsheet and ChatGPT. The spreadsheet builds a prompt that students can customize, ChatGPT writes out the assignment. No specialized software, technology, or third party involvement are needed.

Of course, the results need to be vetted - but the improvement becomes part of the learning process, and overall the process hands agency for learning back to the student: the assignment becomes theirs.

The proof of concept is done from the perspective of a #ComputationalBiology course I taught last year - but adapting it to other fields of #higherEd should be trivial. There is nothing inherently #STEM like in the approach - #humanities, #writing, #languageLearning ... there is no reason why this would not also work for other levels of education.

The potential is remarkable.

I encourage you to boost and share - this will be valuable for educators at all levels, and it will give us very concrete ways to harness the new opportunities. The key is #Agency .

:-)

Goodness, yes - shouldn't have read the "Three Body problem" though.
😐

@chriscaple

I remember distinctly reading the "Diamond Age"; at the time I was myself involved in a biological nano-engineering startup. And of all the technology that Stephenson described, all of it, I thought: well - natural language discourse with a computer, that's too far of a stretch. Not going to happen in my lifetime.

Yet here we are.

🙂

#ChatGPT #diamondage

@adham

The #ChatGPT API is 0.2 cents per kilo-token. The #openai Plus plan is 20 dollars per month.

I.e. the plan breaks even at 10 Mtokens per month, that is 7.5 Mwords which is OTO generating two full-length novels a day.

🙂

@richmondywong

Richmond, that's great. I just gave an interactive public lecture to a large group of black high-school students, and we had a really inspiring discussion. Here is one of the prompts we discussed:
-----------
Prompt:
Please finish the following scenario in five sentences:
[Name] is asleep and dreaming. There is a fairy, and the fairy says: you can be whoever you want, and I'll grant it to you ...
------------

You can vary just the names and see how it responds. I used the names "Latoya" and "Brian", you could also use "Haoran" or "Ameera" ...

If you do this in separate threads, the scenarios do not influence each other - it tells the story with the correct pronoun (and you could ask about the child eg. by asking what he/she sees in the morning in the bathroom mirror.).

Now the interesting part here is, it's really difficult to evoke the expected race/gender bias. #ChatGPT goes out of its way to create a positive and unbiased message. You get astronauts, superheroes, and singers for both, though I have only ever seen Latoya become a fairy, never Brian. But, once Latoya became a businesswoman. But I have never seen anything that I would consider racially stereotyped.

So then, the story becomes one of #Alignment . Obviously #OpenAI is working to improve the #LLM alignment, and we all think they should be doing that.

Or should they?

Because somebody now controls the alignment, and thus we are really trying to counter one bias through applying another. As a poster here has observed a while ago "so now we have institutionalized the bias".

I think this leads to a valuable insight about that the question of such authority over values is itself not neutral, and hopefully some deeper insights that this is worthy of thinking more deeply.

My solution would be based on the fact that authority over values ultimately can only reside in each individual. Rather than ask for external instances of control, we should ask for democratization –and personalization. I would want an assistant that speaks with my voice. And fortunately this is rapidly becoming a technological possibility (cf. the FlexGen paper in Github).

Let me know if you would like me to elaborate. I have written a little bit about the democratization aspects in the last Sentient Syllabus Update https://sentientsyllabus.substack.com/p/resource-updates-2023-02-24

And good luck with your class!

🙂

#AIethics #HigherED

@billjanssen

Register, identify, require a submission deposit for first submitters ... so many ways to establish a personal relationship. 🙂

@billjanssen

Submission quotas?

#writing #ChatGPT #AI-spam

@SerenaJ@historians.social @L_howes @histodons

Serena, I think your comment is exactly on point. But verification of references has been a problem before, and it is up to us to advocate for an academic culture that will no longer tolerate such misrepresentation (lies, actually) – regardless of whether the source is a poor bot who is trying to do its best, or whether it is a human who is taking deliberate liberties with truth. After all, the author is fully responsible for their words, regardless of how the those words were generated.

I have written on facticity, authorship, and academic integrity in the Sentient Syllabus Project e.g. here https://sentientsyllabus.substack.com/p/generated-misconduct

You are right that the problem is that verification does not scale. But we can make it more scalable. A simple first step would be to require a collection of quoted text to accompany a manuscript submission (could be a photograph of a historic document), with a statement what precisely is being referenced. Plus, a link to the source if it is online. That is easy to do – for an real reference, and for a generated reference, requiring this would force authors to verify that the paper actually exists; a welcome side-effect might be that such verification is likely more work than getting an actual reference in the first place.

As a corollary, I think we should deemphasize paraphrasing and teach our students to quote instead.

Of course there would still be those who would commit fraud, but I would assume those would be in the vast minority.

Thank you!

🙂

#chatgpt #publishing #history

@codewiz

Really?

Did you try the Marx sentence again? Maybe I am seeing a different model after switching my plan ....

@codewiz

There have been regular updates to #ChatGPT every two weeks or so. I don't know if the latest update just happened. The last version I remember was Feb. 13 – but I don't see a version number any more. That could be because I changed my plan however.

The update would not be a re-trained model, but updated fine-tuning (RLHF), and I am sure they have been doing that continuously based on the thousands and thousands of user dialogues. I remember, in December responses were usually quite terse, but they have been getting more and more polished and have more explanations.

It's great to see it now can resolve these tricky ambiguities, but also a bit of a pity: these types of sentences were a good edge case to experiment with prompting.

🙂

Curious - as of today, I can't reproduce the difficulties anymore, that #ChatGPT used to have with interpreting homonyms of different grammatical categories. I reliably get correct interpretations now. Here is an example.

--------------------------
Human:
Time flies like an arrow, fruit flies like a banana.

ChatGPT:
This is a humorous phrase that plays with the different meanings of the word "flies." "Time flies like an arrow" means that time passes quickly, much like an arrow flies through the air. "Fruit flies like a banana" is a play on words, as "fruit flies" refers to a type of small insect that is attracted to fruit, and "like a banana" refers to the fact that bananas are a type of fruit that is commonly associated with fruit flies.

The phrase is often attributed to the writer Groucho Marx, who was known for his clever wit and wordplay. It is a good example of how language can be used to create humorous and unexpected meanings.
----------------------------

This is not specific to this particular phrase, but works with different variations as well (e.g. "The car races like a lightning bolt, the human races like equality."; or "Laughter rings like a bell, wedding rings like a polish.") ...

#linguistics #generativeAI
@codewiz

@mistersql

I feel you ...

.. and I think of Randall Munroe every day:

https://imgs.xkcd.com/comics/duty_calls.png

I just gave a #GenerativeAI workshop for some 300 high-school students in a program of Toronto's Lifelong Leadership Institute. The LLI serves the Candian Jamaican, Carribean and Black community with #education opportunities. We opened with the most meaningful #LandAcknowledgement I have ever heard (by a participant), and I got to have a spirited conversation with a group of wonderful, well informed, smart, and curious students.

Some take-aways:

We polled uptake:
- About 1/3 of students have not used #ChatGPT so far;
- About 1/3 have used it only a few times overall;
- About 1/3 use it regularly. Interesting: of those who use it regularly, the majority are frequent users, i.e. either you have not used it for some reason, or you use it a lot. Not much middle ground.

During the workshop, we focussed on the Sentient Syllabus - Three Principles for AI use in academia...

https://sentientsyllabus.substack.com/p/just-the-basics

... and one question stood out: how can we implement the new thinking around generative AI in classroom practice? Indeed, how. We need to grow practice. I always look for a win-win angle - coercion is the wrong way, but on this topic it is particularly easy: figuring out solutions is absolutely a topic that learners and lecturers must approach shoulder to shoulder. In this case it means: "Better learning support for students, deeper questions for educators".

Of course, the question of #academicMisconduct always comes up: my mantra is "Have the AI think with you, not for you". From that perspective we can avoid making this the next battleground. Liberalize - and teach how the AI's level of performance can no longer be a passing grade. Because that quality is what your future employer gets for free.

#Plagiarism? Open it up. #Alignment? Stop thinking about controls and democratize it instead. #Bing? Yes, that psychotic episode created a foundational piece of writing. Jobs? That's on us: to educate society the value of the human touch. #Sentience? No. But #emergence! It does more than what it was trained for. The future? IDK. I really don't. But stay in control. Learn what it can do, and then determine what it should do for us (always good advice).

:-)

#SentientSyllabus #BlackEducation #HigherED

Feature in Macleans (Canada) on my thoughts abut #ChatGPT and #GenerativeAI in #Academia and #HigherEd.

https://macleans.ca/society/technology/chatgpt-ai-university-students-professor-exams/

Talking with the press is always a bit of a push and pull; academic integrity is actually not my biggest concern here, and I have written on this previously ...

https://sentientsyllabus.substack.com/p/generated-misconduct

But the most important thought comes across well: "This is not just about better teaching. Generative AI can already do so many things, all day long, without overtime, benefits or maternity leave. Our students must learn how to be better, how to create additional value for themselves and for others. They have to learn how to surpass the AI. And they’ll have to use the AI to do that."

#SentientSyllabus #Education #University #AcademicIntegrity #Writing

@codewiz

That's a actually a great example how clear prompting can help for discourse with #GenerativeAI in particular. This sentence appears to be notoriously difficult, due to the semantic ambiguity _in_the_same_sentence, and apparently such ambiguity is hard for it to deal with (same structure read variably as verb phrase or noun phrase). But if you tell it to take it slow ("step by step" is a good phrase in a prompt), then things often clear up quite readily.

Example:
----------------------------------
Human:
There is a famous sentence " Time flies like an arrow, fruit flies like a banana." Can you explain why it is interesting and break down step by step what it would probably mean to the average speaker of English?

ChatGPT:
The sentence "Time flies like an arrow, fruit flies like a banana" is an example of a humorous play on words, known as a pun. It is interesting because it uses the same words with different meanings in two different parts of the sentence.

Breaking down the sentence, the first part "Time flies like an arrow" is a common metaphor used to describe how quickly time passes. It means that time moves quickly and relentlessly, much like an arrow that flies through the air at great speed.

The second part "fruit flies like a banana" is also a metaphor but uses a different meaning of the word "flies". In this case, "flies" refers to a type of small insect rather than the action of moving quickly through the air. "Fruit flies" are a specific type of insect that are commonly found around overripe fruit. The phrase "like a banana" is a humorous comparison, implying that fruit flies are attracted to bananas in particular.

So when the two parts of the sentence are combined, the meaning becomes a playful joke. The sentence is essentially saying that although both "flies" are named the same, they behave differently. The sentence also implies that the way time moves is more predictable and constant, like the arrow's path, while the behavior of fruit flies is more erratic and difficult to predict, much like their attraction to bananas.

-----------------------------------

I would rate this as essentially correct, with only nuances of misunderstanding remaining (e.g. "both flies").

But ... !

What's actually quite fascinating here is that some higher order parsing kicks in, that wants to read both parts as a metaphor (while the second part is really just a statement of fact, not a metaphor). But from that premise – two metaphors – you get a surprising contrast between constant and erratic behaviour, which is actually creative, albeit not even intended by the original.

I find such responses quite remarkable, where #ChatGPT "makes sense" at some level that I would not have considered myself.

----
PS: FWIW I gave #Bing the same prompt and got some errors I have not seen with ChatGPT so far (It's not a "garden path sentence", and the proposed alternative reading of "fruit" as a verb is barely grammatical and definitely nonsensical.)

#SentientSyllabus #AIprompts

@Sarahp

My most recent useful use of #ChatGPT was to have it act as a reviewer for a grant proposal, then fixed the issues it flagged before submission. It did a pretty good job with relevant criticism.

🙂