00:00:06.980 - 00:00:30.930

All right, good afternoon, everybody. I see we have folks coming in from the waiting room. So welcome, welcome. All right. Just want to make sure we have a critical mass here before I jump in. OK, so, um, thank you all so much for being here. Uh,

00:00:31.000 - 00:00:48.090

this is our first event of the calendar year 2025 in our Biddle speaker series. My name is Genevieve Tang. Um, I'll be your host for today's session. Uh, the Speaker series exists to showcase emerging and impactful scholarship on all aspects of legal information, and it is

00:00:48.090 - 00:01:04.650

my great pleasure to introduce our speakers today, Sarah Ryan and Kassowski, who will speak on their forthcoming chapter, Artificial Intelligence and Machine Learning in Law Libraries, which will appear in law library practice later this year. I have had the pleasure of learning from both of our

00:01:04.650 - 00:01:16.730

speakers today, and I know we're in for a treat. And their professional accomplishments are also very substantial, as I think many of you know. So I'm just going to offer a few highlights and then we're going to get into our conversation. Sarah Ryan is director of

00:01:16.730 - 00:01:32.300

the Law Librarianship and Legal Informatics Program at the University of North Texas. She previously served as head of empirical Legal Research Services at Yale Law School and is co-editor of the Journal of Library Resource Sharing and a past editorial board member of Law Library Journal. Sarah

00:01:32.300 - 00:01:51.279

is an award-winning author and currently researches in the areas of legal technology, access to justice, equity in higher education, and library science pedagogy. Castlekowski is a veteran, librarian, gamer, teacher, Latina, techie, comic book nerd, and empiricist in perpetual beta. Currently the acting co-director of the University

00:01:51.279 - 00:02:07.709

of Arizona, she is a leader in national and artificial intelligence efforts, including co-founding the Future of Law Libraries Initiative. She was a founding fellow at the University of Tennessee's Libraries 2021 IDEA Institute on Artificial Intelligence. And the only librarian of any type to be selected for

00:02:07.709 - 00:02:24.389

the 20202022 Summer Conference on Applied Data Science held at North Carolina State University. She currently co-leads the UA Making AI Generative for Higher Education Project team and serves on the Arizona Steering Committee of Artificial Intelligence and the Courts. Throughout today's talk, this is my housekeeping bit.

00:02:24.830 - 00:02:35.949

Please feel free to add your questions in the Q&A. I will be monitoring the chat and the Q&A so that Cass and Sarah can focus on their remarks. Uh, we'll have time, a little bit of time, uh, for our speakers to address your questions at the

00:02:35.949 - 00:02:52.729

end of the session. Uh, and I just realized I did not set. I just want to make sure my recording is set. Looks like we are good. Whew. So, um, Automation. Uh, so we will be posting the recording, uh, on our institutional repository page afterwards and

00:02:52.729 - 00:03:11.490

sharing that with registered attendees. Um, so again, thank you all for being here. Uh, and let me turn it over to Sarah. Thank you, thank you, and let me share my screen. Oh, just like when I'm teaching, it's gonna take me like 3 tries, so, you

00:03:11.490 - 00:03:35.139

know, you all know the drill. Just, just take a little power nap while I get my screens working. There we go. All right, and we're almost at the 3rd try. How's it looking? Did I get it? You did, you did in fact. Yay! So I'm so

00:03:35.139 - 00:03:49.460

excited to be here with cast, um, the real brain of this operation, uh, we're starting with history, and, um, uh, you know, I was the lead on that section of the chapter, so I'm gonna start us, but you'll get cast soon, and we really appreciate the,

00:03:49.470 - 00:04:09.440

um, the invitation, Genevieve, and the opportunity to, um, share our forthcoming work. So we're gonna talk about artificial intelligence and machine learning in law libraries. And we'll start off with the history portion of our forthcoming chapter on managing disruption. So, we start our chapter off with

00:04:09.440 - 00:04:24.140

a history lesson to reinforce that AI and machine learning are both ruptures and disruptions, of course, but also that they come out of the great work that we've been doing for decades, and there are a leverage of that work. So many of you are familiar with

00:04:24.140 - 00:04:45.059

Thomas Kuhn's, the Structure of scientific revolutions, and you'll know that one of the main premises is that the work that's happening in labs, and other places like libraries. This day to day grinds and the continual effort to solve problems, that's what leads to breakthroughs, and that's

00:04:45.059 - 00:05:00.440

what we're seeing with artificial intelligence and machine learning. There were efforts since the middle of last century to solve problems, and to enable people to access information and to work together. And that's what led to these new disruptive technologies. And so many, so much of this,

00:05:00.540 - 00:05:16.859

you all know, came through our libraries. And so we start our chapter off with um this history information as a hook for staff who are reticent about AI but also just a celebration of the law librarians who let, who, you know, paved the way for AI.

00:05:17.570 - 00:05:34.850

And let me give you just a few library and law library touchstones. So you've read about early pioneers of AI such as um Alan Turing, John McCarthy, you probably come across their names. But there was also an important physicist, Meyer Kessler, who was at MIT running

00:05:34.850 - 00:05:51.600

their technical information program. And one of the things that occurred to him was that one of the best incubators of these ideas in the 60s was the MIT libraries. And so he got himself named, this is a physicist, he got himself named the associate director of

00:05:51.600 - 00:06:09.239

the MIT libraries. And one of the projects that he worked on with the librarians was to make physics physics journals searchable and retrievable across libraries, across the country, and so we have this schematic of how all of this is gonna work. And then we have an

00:06:09.239 - 00:06:29.420

early console. Um, and so we crosswalk that work in our chapter, um, over to law librarianship, and we ask, um, when did law librarians start getting interested in AI, machine learning, all the roots of what we're seeing today. And again, we found our way back to

00:06:29.420 - 00:06:52.630

the 1960s with the Law Library Journal reporting on a major investment by the Council of Library Resources, um, to explore. Um, the future of, um, computerized storage and retrieval, but even if you go to the end of this little, uh, current comments, artificial intelligence. And then

00:06:52.630 - 00:07:12.420

we found a 1970s meeting at the Chicago Association of Law Libraries, a seminar on library automation, and after that conference, LLJ published nearly 20 of the papers from that Chicago conference in a special issue on technology. And you can see in the introduction to the issue,

00:07:12.839 - 00:07:28.880

you know, all these acronyms that were just flying at librarians back then from light to Cal, um, but so much was happening in those days, and law librarians were, were making a lot of sense of it and educating each other. Um, and we found more special

00:07:28.880 - 00:07:45.850

issues. We've got Morris Cohen telling us that it's, it's our duty as law librarians to stay up to date on this, um, so that we're making educated. Decisions about what to implement in our libraries, um, and, and so we're in the know. And then, um, in

00:07:45.850 - 00:08:09.010

the history research, the most exciting part for me was 1979. 1979 was amazing. The AALL conference theme was legal information to the people. And then there was a White House conference on library and information services that yielded resolution A12 on the need for a national policy

00:08:09.010 - 00:08:29.529

to ensure equal access to necessary basic legal information to all people. That's 1979. Now we know the history from there. No federal agency answered the call to oversee open access legal information. The Law Library of Congress does what it can, uh, but we don't have an

00:08:29.529 - 00:08:45.169

agency tasked with making sure that all of the legislative materials in our country are open access and easy to find and in the same format, or that Pacers is not behind a paywall. And we also know that vendors moved into this space, into this leadership vacuum.

00:08:45.780 - 00:09:03.250

But the point of starting our chapter with history is that we're at another moment, like the 1960s and 1970s, where the work that everybody's been doing for decades has led to disruption, and we get to ask again, what does the future of legal information access look

00:09:03.250 - 00:09:20.820

like? Um, and I'm excited by that, and Cass is excited by that. And so now I'm going to turn it over to Cass for an overview of AI and ML fundamentals. Thank you, Sarah. So once we sort of laid that history, we moved on to try

00:09:20.820 - 00:09:36.969

to create a foundation of understanding of machine learning generally and some concepts. My goal is not in this presentation to try to provide a class on that. It'll provide some highlights so you know what to expect in the book, but I really, we both decided to

00:09:37.340 - 00:09:55.549

preserve most of the time for the in practice part of our presentation, which will come after this. So next slide. One of the big things this section tries to do is provide you enough information to understand how you might even approach the idea of an AI

00:09:55.549 - 00:10:08.385

project in libraries, and part of that is understanding what the question is that you want to ask, right, because without understanding the question you want to ask, you don't know what the data is you want to need or the machine learning model you might apply. Or

00:10:08.385 - 00:10:27.465

the program you might use. Here are just some images actually from the textbook you'll see showing different types of machine learning applications from very classic logistic regression to guess pricing to sorry, linear regression to guess pricing to logistic regression to kind of say yes or no

00:10:27.465 - 00:10:41.859

about a particular thing like is someone going to pass the exam or Filtering or recommendation systems and the different forms of that, but each of these depend on the question you were asking. So if you're asking things like how to adjust staffing to meet higher demands

00:10:41.859 - 00:10:54.739

over time, you need to predict. You know when those demands are going to be high and how many people you need to meet them, how to enhance the acceptance rate of faculty or attorney scholarship, right? That's going to be another table. You might also need to

00:10:54.739 - 00:11:09.809

efficiently create alternative text for a collection of digitized images and even in special collections, but you have one archivist and not enough time to have them write alt text altogether. So once you kind of an idea for trying to approach these next slides. You want to

00:11:09.809 - 00:11:23.770

think about what the development process looks like. And so the first stage of that process is going to be gathering and preparing the data. The second stage is going to be selecting the algorithm, that means like the method you want to apply. So linear regression is

00:11:23.770 - 00:11:34.864

the method, but then you're going to have to do what's called trading the model. Creating the actual model, which is drawing that line across your data. That's 3. So that 3 is going to be training and evaluating that model. So once you say I'm going to

00:11:34.864 - 00:11:48.344

apply linear regression, you need to actually figure out what your line looks like on your own data, right? So that's the difference between an algorithm and a model. And this chapter does a lot of that too, trying to help you suss out what the appropriate language

00:11:48.344 - 00:12:01.349

should be despite what it may or may not be in articles and Um, journal height or things like that. And then 4 is going to be talking about what, what does it look like when you manage to get through these different steps and then you have

00:12:01.349 - 00:12:14.340

to put it into practice, right? We want to talk about that as well. So next slide. We talk about data and gathering that data. It's really important not just what data is available, but what resources you have to engage with that data. So it's one thing

00:12:14.340 - 00:12:29.580

to just take this picture and tell the computer, just FYI this picture has a sheep in it. It's a whole other thing for somebody to have the time or if you have the funding for a really, really, you know. Detail oriented RA to draw the line

00:12:29.580 - 00:12:42.260

around the sheet so that you tell the computer this is the exact part of the picture that has the sheet. Those are two different things, and that changes your data, which changes the model that you can, the results of the model that you can expect once

00:12:42.260 - 00:12:55.390

you train it. Part of once you've collected this data and gotten the data ready is going to be engaging with the data in the next slide. Because if we do a superficial job of engaging with the data, we might think, oh look, here's our prediction. We

00:12:55.390 - 00:13:08.669

have this sort of positive relationship going on between these two factors, but if we don't engage well, we're not going to be able to shine light on the color of our data. Next slide. To really show that that's a fallacy, right, so this is called Simpson's

00:13:08.669 - 00:13:23.599

paradox. It's the idea that a trend you might see in data disappears when you start to look at other facets for whatever reason. And so what you thought was a positive relationship actually turns out to be the exact inverse, and you can't do this until you

00:13:23.599 - 00:13:40.510

actually meaningfully and intentionally engage with your data. If you just threw this data into a machine learning model in the previous slide with just those two factors, you would get a poor prediction. So, next slide. Once you have your data and you've figured out, you know,

00:13:41.210 - 00:13:53.460

what exactly is the structure of it, what your goals are with it, you need to kind of pick your algorithm, and part of that is going to depend, like before, on what your data looks like. If it's Structured and well tagged, if you had a lot

00:13:53.460 - 00:14:08.659

of time and resources, so your data is very clear about this is my set of alumni, this is my set of space as opposed to people, then maybe you can have the data set really clear enough to do a classification because you're telling the computer what

00:14:08.659 - 00:14:23.390

to expect. Clustering, however, is for unstructured data, and that's for when you have that stuff that's I theoretically know. How to tag these, I don't have the time, so I'm just going to go and ask the computer to find some patterns, and then I'll probably label

00:14:23.390 - 00:14:41.229

them after the fact, right? So that clustering asks the computer to find those patterns for you, and the beauty of this is is. They can honestly both come out, those models that come out of these choices can result in the same output, but they depend on

00:14:41.229 - 00:14:54.059

your data and your resources. Next slide. And then last, when you deploy it, it's really important that these phases are where you're challenging them constantly. You don't just put it out into the universe and never engage with it again, whatever it is. So when we talk

00:14:54.059 - 00:15:09.934

about these models upcoming, when we talk about these next steps and the ways that you might input these things, it's really important that you evaluate them. So really, really good for sandwiches, then a little less. For flat panini pressed events, right? Maybe this isn't exactly the

00:15:09.934 - 00:15:24.695

computer vision model you might use in the library. I like food, so I always want to focus on food, but right, if we don't look at this and we don't think about how it's applied, right, there might be more negative impacts than missing my food. Next

00:15:24.695 - 00:15:44.789

slide. So our first practical application of this, right? I reference services. We're gonna talk about different aspects of the library, we're gonna focus on reference now. Next slide, the most, I think, popular example of using machine learning or any sort of artificial intelligence system in reference

00:15:44.789 - 00:16:08.479

services is going to be the chatbot. This has come a long, long, long, long way from a classic artificial intelligence, um. Model of just a decision tree, if this, then that, to more data-driven models that are able to find similarities in language between different types of

00:16:08.479 - 00:16:30.359

questions to map to a created answer, to now even generative AI systems that can pre-produce answers to these um questions without ever even creating the back-end knowledge base. Now, next slide. Where you apply this and how you use it is going to depend on how much

00:16:30.700 - 00:16:48.059

you want to engage with and interfere with the potential outputs, so things like hours and locations are really clear and clean and things to put in a chatbot to try to triage out some of those smaller questions. But whether you want to have your chatbot answering

00:16:48.059 - 00:17:04.479

complex legal reference questions from students about papers that they have for class. will depend on whether one, you're just using the model in a way to guide them to resources and it's been trained to just kind of say here this sounds like some resources that correlate

00:17:04.849 - 00:17:18.839

to the terms in your in your query versus here's a sample of what you might draft, and then all of a sudden you're providing like preliminary research for their open research motion or something to that degree. You're just gonna have to work that out with your

00:17:18.839 - 00:17:35.040

team. But I think the more interesting things with reference happen when we think about enhancing ourselves in our own work. Next slide. And part of that's going to be about looking back, so looking at our old reference queries and asking, remember that that I could tag

00:17:35.040 - 00:17:46.729

this all day if I want to, but I don't have the time because I barely have the time to answer the reference questions? Well, ask the machine to see what patterns it's finding in reference queries over time. And it can help you start to see which

00:17:46.729 - 00:17:59.780

types of queries are you getting a lot of. Are there ones that maybe we could have some better template answers for that we could map to a knowledge base so we can get those out faster? Or is there an increase in this type of question coming

00:17:59.780 - 00:18:12.390

from faculty that maybe we need to start training our staff to be better prepared to answer, or do we need to enhance our collection because a lot of people all of a sudden are asking about mining law because we have 8 new mining law courses. Whatever

00:18:12.390 - 00:18:29.660

that is, mapping these patterns in the reference query historically that we've gotten can help inform our work going forward either to make it more efficient or to enhance other services at the library. Next slide. Um, some systems can be used to triage the incoming emails, so

00:18:29.660 - 00:18:44.130

if you have a faculty service liaison model that has different librarians for faculty, Sometimes they still message the main queue, right? Ours do. Yours probably do too. It could easily forward it to the right person, tag it to who it should go to based on who's

00:18:44.130 - 00:18:56.880

coming through or the type of question. Maybe my faculty would normally come to me, but this one's asking about UN regulations, and that is Marcelo 100%. I'm not touching that with a 10 ft pole. He is just far more skilled. It'll get the answer out 10

00:18:56.880 - 00:19:13.989

times faster. Next slide. But I think the biggest thing. With reference services is going to be the way we engage with students and faculty on their own AI enhanced projects, which requires us to be knowledgeable enough to support that. So this is the on the books

00:19:13.989 - 00:19:30.050

Jim Crows and algorithms of resistance at the University of North Carolina. And this entire project digitized entire sets of session laws and then created a training set of Jim Crow laws to try to then have that model that it trained search for additional Jim Crow laws,

00:19:30.060 - 00:19:49.180

and they made that public. Next slide. Such that they created an entire GitHub with these models, and here you can see model selection because they had to figure out which one performs better, training codebook, training set, versions 1 and 2 in the supplement, there's just this

00:19:49.180 - 00:20:04.464

entire process that goes. On with these projects and you may not be able to engage with all of it, but understanding machine learning and AI generally will help us be involved in these projects because the law librarian here helped them understand statutes and session laws sufficiently

00:20:04.464 - 00:20:21.189

enough to model the infrastructure and understand what they're putting in data wise into these models. But maybe they could have taken the next step if they happen to be more interested. Whatever we do with these projects, this knowledge next slide is going to really help us

00:20:21.380 - 00:20:36.890

find information for our faculty, our students. This is the hugging face interface, which is just a sea of pre-built open access models for different work. You can see here for audio to text, images, all sorts of things, document analyzing. We don't know what even this is.

00:20:36.969 - 00:20:58.989

We can't really talk to our students about going here and finding it. Next slide. And more impactfully for me. is in faculty reference interviews, we can really take the moment. To intercede in potentially problematic research. This project by a North Carolina, Eastern North Carolina University professor

00:20:59.609 - 00:21:19.290

took a bunch of YouTube videos of individuals going through HRT. And then Created a data set to show the before and after so a model could predict when someone was taking HRT. Their premise was this would help in case terrorists decide to take HRT to evade

00:21:19.750 - 00:21:41.619

detection through facial recognition, but the impact is so much more problematic. And if we have this knowledge, we can have these conversations with faculty who have the desire to do something cool. But maybe Not something they're considering all the implications of, Sarah. Thanks, Cass. And um

00:21:41.619 - 00:21:57.829

the next section, uh, dovetails right away into research instruction topics. Um, and our first question was, what are we already doing? Uh, because again, one of the premises of our chapter is that we're, we're already experts in this, that, that we can look to what our

00:21:57.829 - 00:22:16.459

community to see, um, the future of AI and ML uh in what we're doing. So, of course, we find new lib guides on, for instance, chat GPT and generative AI and legal research from the University of Arizona Law School. Um, and remember that year or two,

00:22:16.560 - 00:22:35.770

when we all got upskilled. So when we started looking, you know, our, our community was doing a lot of training and education over the last few years, uh, to help us all really quickly, um, upskill our instruction. Um, then we all started, or, or many of

00:22:35.770 - 00:22:48.930

us who are teaching legal research started assigning these, and this is from, uh, one of my legal research courses. And, and so the next thing we thought about in our chapter was, um, you know, we all need to grapple with, um, students' responses to this, and

00:22:48.930 - 00:23:01.030

that's what we've really been doing the last year or two, too, is to say, hey, this is, I'm assigning you chat GPT and you may have a problem with that. Um, how do we dialogue about that? How do you get back to me as your instructor

00:23:01.030 - 00:23:19.609

and, and tell me what problems you have with this conceptually and operationally, because I, I know that this is not traditional research instruction. Then for our chapter, we also wanted to do a lit review to get ideas for new courses, workshops, units. We said, if we're

00:23:19.609 - 00:23:38.550

teaching to the scholarship around generative AI, the legal scholarship, um, what would that mean? Uh, so we searched timeline in the law library journal, um, date restricted to 2016 and later. Sorted by the number of times, that articles were accessed, uh, reviewed and read 30 of

00:23:38.550 - 00:23:55.180

the articles, and then, you know, selected the article. Uh, if it was in the top 10 results from a highly cited author or added a new topic, and this gave us a list of 4 areas that, um, are not going to be surprising in some ways

00:23:55.180 - 00:24:08.989

to many of you, but, but these are the things that we can enhance in our teaching over the next couple of years if we want to teach to the scholarship. The first one, of course, intellectual property. Um, and you'll see in the law journals that, um,

00:24:09.109 - 00:24:31.209

faculty are outcrying, uh, you know, how slow the USPTO has been to deal with, uh, the IP issues surrounding AI. And so I got this, um, you know, screenshot showing the Copyright Office starting a new initiative to look into copyright issues related to AI in 2023.

00:24:31.750 - 00:24:44.250

Um, you know, after all of us have been upskilling, USPTO has finally decided that this might be an issue too, so they might be a little behind. Um, and our lit review led us to some of the topics, some of the subtopics that we could focus

00:24:44.250 - 00:25:08.300

on in new courses or workshops or units, um, innovation, generative AI, um, authored works, copyright, First Amendment law, patent ownership, and what are we gonna do with human-like authors? The second domain we found was privacy, human rights, and democracy. And in this domain, we found a

00:25:08.300 - 00:25:28.319

foundational article by Carl Mannheim and Lyric Kaplan that according to Hines, scholar check has been accessed more than 2000 times and cited more than 60 times in just a few years. And Mannheim and Kaplan describe AI as the most disruptive technology of the modern era, and

00:25:28.319 - 00:25:49.530

they lay out a bunch of threats, um, that have all subsequently been examined in other articles, new forms of surveillance, diminished autonomy, new risks to elections and democratic institutions. Violations of transparency, accountability, and fairness. Um, so their article is about risks. Um, and we could teach

00:25:49.530 - 00:26:09.810

to the, this list of risks in a, in a workshop or course. And then there's some, um, other things we found in, in this constellation of law review articles, deep fakes, AI is evidence, or, um, even the privatization of governance, um, in the AI sphere. The

00:26:09.810 - 00:26:29.969

third domain we found was business law. And here again, we found a foundational article in Julie Colen's Law for the Platform Economy. Um, and the citation counts were interesting. Remember that the last foundational article was accessed 2000 times and cited 60? Well, Hines says that, uh,

00:26:29.979 - 00:26:50.760

Professor Cohen's article was accessed 200 times, but cited 120, so I don't know what's going on. But, um, nevertheless, it's a foundational article in our lit review. Um, and Professor Cohen is, um, arguing that, ah, the platform economy, ah, isn't just a new business model, um,

00:26:50.800 - 00:27:09.790

or even social technology or infrastructure, it's an emerging informational economy, um, and she sees pros and cons to this new economy. Well, also in business law, um, in addition to the platform economy, we get some other topics that we could, uh, you know, ramp up our

00:27:09.790 - 00:27:31.180

teaching around blockchain. AI and anti-bias intervention. So, you know, when you have AI screen 500 resumes, what kind of bias are you getting in that screening? Um, and then, um, you know, just the idea of too little data being collected, uh, by government on Facebook and

00:27:31.180 - 00:27:45.060

Rory Van Lew's article, but of course, on, um, many of these emerging AI companies as well. And all of this got me thinking about um a book that many of you have read and incorporate in your teaching, and that's, you know, it seems to be always

00:27:45.060 - 00:28:01.630

relevant and relevant again, that's Sarah Landon's data cartels, and so, so much of this business law research, we could come back to her book and integrate that too. And, you know, overall, the business law research was interesting, because while there are some critiques of AI in

00:28:01.630 - 00:28:17.030

it, there's far more excitement about AI than in um the collection of articles that we're dealing with like the downfall of democracy. They weren't, they weren't really excited about AI, but the business law folks are going to give AI a chance, or perhaps they're just more

00:28:17.030 - 00:28:35.229

realistic on the fact that um how we do business is changing. Um, so throughout these articles, of course, we saw state laws in, in all of these other domains. So, we looked at state law resources, um, and the National Conference on State Legislatures was tracking AI

00:28:35.229 - 00:28:54.199

on, on this page until about last September. The Council of State Governments did a big project on artificial intelligence in the states. And then we also found the Brennan Center, um, continuing to look at artificial intelligence regulation. So these are all, you know, resources for our

00:28:54.199 - 00:29:13.119

teaching and directions that our teaching could take. And uh the next section, um, I was also just like I was excited about 1979. I was really excited about uh researching access and technical services, and maybe it's because I secretly want to be in TS, um, but

00:29:13.119 - 00:29:34.630

really, access and technical service departments, I think, I, I would argue are, are just where the brain trust is on AI and ML. And that's because, um, you know, they are the units that have had to change the most over the last decades, um, and that

00:29:34.630 - 00:29:57.119

access and technical services units have already accepted that texts are data. That machines are often the first people or whatever, patrons to read texts. And that computers have to shoulder some of the work because funding for librarians has not proliferated like information has. So let's talk

00:29:57.119 - 00:30:17.689

about Texas data. When researchers view our collections as data, that changes our collections. When texts are data, patrons want to mine everything we have. When data sets are texts, then we become the managers of data sets. Um, and so, um, TS and Access have been grappling

00:30:17.689 - 00:30:39.479

with this for years, and they've been using OCR, they've been using NLP to automatically translate texts, they've been using automatic indexing, uh, using expert systems. Um, and, you know, authors writing about TS and access work, say, you know, newer developments in AI are, are both going

00:30:39.479 - 00:31:06.030

to Um, add new wrinkles, um, but also just increase the specialization of these departments, um, and, and the data skills of these librarians. Next, um, Moving my Zoom things around, there we go. Um, uh, Access and TS folks know that machines are our readers, our patrons.

00:31:06.189 - 00:31:22.949

So, you know, we transitioned to federated searching and figured out how users search, um, and what kind of metadata we needed. And then we developed modern metadata standards, uh, and governance procedures. Um, and this has paved the way for, you know, our, our library systems, which

00:31:22.949 - 00:31:38.469

assume that a machine is going to be the first level reader, not a human, but a machine is going to get to our collections first. Um, that, you know, when I think about my students, for instance, they're sending a federated search to look for materials. They're

00:31:38.469 - 00:31:58.800

not going to the library, um, and looking for materials, they're sending a machine out to do it. Um, and that machine is interacting with the collections first. And we see some fascinating projects like this one at UNT where they're um partnering with computer scientists to uh

00:31:58.800 - 00:32:16.449

use machine learning to improve automatic detection of um individual documents and web archives. And then one of the librarians said to me in explaining this project, you know, you have these huge archives and these precious individual documents get lost. Um, but what if we could harness

00:32:16.449 - 00:32:34.829

the power of AI and machine learning to find a single document within 100,000 document collection in a, in a better way. Um, and the final thing that our Access and TS departments have understood for a long time is that computers and now AI must do some

00:32:34.829 - 00:32:55.439

of the work because librarians cannot possibly manage every single resource and bit of data. And so we see OCLC products like WorldShare using artificial intelligence to suggest subject headings and to enrich metadata. And we see ex Libres Rappido, um, simplifying parts of the interlibrary loan process,

00:32:55.770 - 00:33:14.989

such as suggesting the best lender for an item. And, um, the final thing, one of the final things I'll talk about is this scandalous article with this um unscandalous title. The, the title of this article is Troubleshoooting Electronic Resources from an Innerlibrary Loan Perspective. Um, but

00:33:14.989 - 00:33:37.229

what the article is really about is the collapsing of traditional distinctions between access services and technical services. And, and the author of this wonderful and scandalous article, um, argues that, you know, as these, um, Units continue to to grapple with Texas's data and machine readers and

00:33:37.239 - 00:33:55.329

and all of these facets of modern lives, it may make less and less sense to have access MTS be separate departments and rather just combine them and have um You know, data, data technical expert driven departments, um, and, you know, that's food for thought for all

00:33:55.329 - 00:34:16.479

of you, but I, I found it fascinating and exciting. And now back to cast for library space. So, It would seem, right, with all this artificial intelligence, we're talking about reference services, we're talking about, um, you know, access and technical services, we'll be talking about training

00:34:16.479 - 00:34:37.919

here shortly, like the space. Doesn't seem to come into conversation a lot, but it's so important both because libraries are always grappling with how to create meaningful space, but also to protect the space we have and to continue to serve our mission with our space, and

00:34:37.919 - 00:34:56.260

it is kind of interesting because there's always been a question, next slide on what does the library do with regard to artificial intelligence. To try to support this work that's happening, how do we create artificial intelligence spaces? Well, this is the artificial intelligence lab at the

00:34:56.260 - 00:35:11.709

University of Rhode Island, and it's really interesting because they do all sorts of things in there. But if you just were to look at it from the outside, which is what we're doing right here, you would think we're bringing back the computer lab, and to some

00:35:11.709 - 00:35:28.350

degree, to a small degree, maybe. But not with the same purpose. It's the intent that matters. So to some degree there are these few stations there, but what's invisible in this space are the other tools that they have. So first of all, those computers are a

00:35:28.350 - 00:35:46.010

different computer. It's not just a get on and do your homework computer. It's a computer with both the software and the hardware and the Python and our libraries, everything you need to get on and do it, where a student's normal laptop they bring to work may

00:35:46.010 - 00:36:01.500

not be or to school may not be sufficient to run the type of machine learning processes, even basic ones, even a basic model training for a simple R&N would likely be too much for many of our students' computers. And so to create equity, to create access

00:36:01.500 - 00:36:17.439

to this technology, we create the space for them to engage. With these types of tools, but also within this space are things for students to engage with. So they have things like robots and drones and different things that they can use these tools to create, say,

00:36:17.530 - 00:36:34.379

data sets that they might use for some AI projects, right? You might use the drone to do some, some video capturing in some areas and then use an AI model to try to identify landmarks inside of there. But also in space, sometimes it's about creating the

00:36:34.379 - 00:36:53.489

space for collaboration. Next slide. This is our collaboration space at what we call our collab, College of Law Lab, but the, the design is 4 pods right along the side. You're seeing one here in the corner where they can be used individually for collaboration, but the

00:36:53.489 - 00:37:07.110

room can come together to allow people to try to do things in these individual modules and then throw them onto the main monitor. It also has kind of a touchscreen main monitor that allows people to come together and brainstorm. And try to do this kind of

00:37:07.110 - 00:37:23.270

work. It also has really high powered networking like the webcam. All this stuff is really high end so they can engage and bring people on as part of their products for guest lectures or things that students are doing. And what we're doing is we're creating the

00:37:23.270 - 00:37:44.939

infrastructure to support these projects. But what else there is, is going to be that kind of back end, which is that invisible available hardware, which next slide. Here at the University of Arizona is our high performance computing lab, right? And so part of this is us

00:37:44.939 - 00:38:02.879

knowing what exists or working and advocating, collaborating to create this kind of thing, which is just theoretically the HPC is its own fancy computer lab. It is just a room of servers with no monitors, right? It is just servers upon servers upon servers, but they're high

00:38:03.540 - 00:38:24.169

quality major GPUs to try to be able to process the data sets needed here at the university for like our ginormous. Telescope images that we get to process for astronomy and other things, all of our healthcare work, everyone has access to go into and use the

00:38:24.169 - 00:38:35.689

server space to try to do these projects. But not any of the students necessarily know it exists or how to access it, so part of it is creating the space for them to engage, to collaborate, right, so they can get together. So maybe you host a

00:38:35.689 - 00:38:46.050

class in our collab space and you have it on the monitor and you teach them how to do it. Students work together to try to work a project through, but you have this kind of back end so that they can do this kind of work that

00:38:46.050 - 00:39:00.139

they may not be able to do with their laptops. But if they don't have the skills necessarily to maybe go through a terminal and code their way in next slide. There's also this sort of interface, this GUI that has existed for us on our web page,

00:39:00.219 - 00:39:18.050

so Sarah has harped on the things we have done for all time, right? So part of this is just gatekeeping. It's just like guiding people through to the source, teaching them about it, showing them, and even if we don't necessarily all of us have the skill,

00:39:18.129 - 00:39:31.209

you could point to people. So here people point to me, but if I wasn't here, people would point to our data services team at Maine library, right? Many of these places have that. Of course, if You know, you have these types of systems existing in your

00:39:31.550 - 00:39:46.189

space. You might have people running them, and you can point them to those people. There's classes that often engage in these things, but the idea when you talk about library space is both about curating the physical and virtual spaces for people to engage in this kind

00:39:46.189 - 00:40:13.239

of creative work, this next level work. Next slide. But all of it, absolutely every last bit of it comes down to staff. Both our advocacy, but also our knowledge. We have From the beginning of librarianship on, been information experts, and artificial intelligence is simply a means.

00:40:14.879 - 00:40:34.280

To engage with information, right? It is a way to do so more efficiently at times, more accurately at others, less accurately at some, but right, the idea is that we ourselves, just the same way as we have been really good curators of quality information, can be

00:40:34.280 - 00:40:54.080

really good curators of quality artificial intelligence systems, data sets, experiences. Next slide. But we need to train ourselves, and there's so much training available. So many of you might remember the library carpentries, which train on all sorts of skills related to data science and programming. They

00:40:54.080 - 00:41:10.840

released an intro to artificial intelligence for GLAM, which is galleries, libraries, archives of museums. I just love the GLAM, the GLAM acronym. It's great. This is a great introduction, right? That kind of base level knowledge also, so is our textbook or our chapter in the textbook.

00:41:11.070 - 00:41:30.060

But, um, right, and that little base level knowledge. Next slide. You also have the opportunity. There's a growing communities of practice around this site. So AI for Lamb is a group of librarians globally. That are trying to get together on this topic and they quite literally

00:41:30.060 - 00:41:48.600

have reading groups. They have monthly get-togethers over Zoom to talk about some reading that was suggested. They have resources, right? And so getting linked into these communities of practice is going to be key for staying up to date, learning about what you might need to know,

00:41:48.620 - 00:42:07.020

or. Libraries are not about one person doing all aspects of library work. It's about diversification of skills towards the greater good, right? So maybe you're not the person that's going to be the AI person on staff, but staying inside of these communities of practice can help

00:42:07.020 - 00:42:25.669

you realize we need somebody on this who is motivated, right? And then supporting that person through that because now you are also aware of the tools to help them get up to speed. Next slide. We also have more advanced tools. So this is FAST AI's for

00:42:25.669 - 00:42:38.989

GLAM study group. So it's a mix, right? We're taking it up a little bit level, up a level, and so in this we're starting to learn not just about the fundamentals. We're not just trying to understand kind of how things work or get an understanding of

00:42:38.989 - 00:42:53.370

what the, what the concerns are or what people are doing with it, but really trying to understand what it means to make them. Next slide. And many of these proprietary systems have trainings with them because they want you to use them, so they want you to

00:42:53.370 - 00:43:13.030

be good at them. So Google has an entire set of training on their AI models, so you understand the different types of models. Next slide. But if you want to engage in the You know, critical concerns and pedagogy around AI. Fast AI has a data ethics

00:43:13.030 - 00:43:25.090

course, right, also by Fast AI that tells you like kind of what these different things are happening because maybe you're not going to build the systems, but you want to be able to engage with faculty about their projects and say, hey, have you thought about whether

00:43:25.669 - 00:43:44.229

there's any built-in bias or historic problems in the data set you're going to use? Next slide. But when we get to the point where we want to do more, Right, then Google also has this machine learning crash course to really try to help you work through

00:43:44.229 - 00:43:58.110

practically using models for different things. They claim 15 hours. Please don't stick to their 15 hour thing. You do it on your time. Don't feel bad if it takes you 25 or 40. We have real lives. This is how they're selling it to you. They sell.

00:43:58.159 - 00:44:14.870

It's free. But the idea is like, The point is to skill up, not To hold ourselves to some kind of amorphous standard where we feel behind because we are so far ahead of the curve on many of these things. As you learn about them, you're going

00:44:14.870 - 00:44:30.929

to realize that. Most of the things that we have talked about ad infinitum as a community are the things that we continue to talk about as we learn about these. They're just applied in a different way. There's just a line of code that changes this idea.

00:44:31.010 - 00:44:44.209

The same as the Simpsons paradox that I showed you earlier where just engaging with it thoughtfully changed everything. We have that critical engagement skill and so learning this stuff, you're going to see those moments where, oh yeah, I would have caught that. Yeah, of course you

00:44:44.209 - 00:44:55.790

would have because you're brilliant, but right, we want you to think about these things as you learn it, but there's so many tools out there, and I have more, but not enough time, so I'm gonna pass it to Sarah to wrap us up before any questions.

00:44:57.979 - 00:45:12.899

All right, thanks, Cass. So here's our big takeaways, um, and, and kind of what we've been, um, emphasizing the whole time is, yes, this is a moment of change and disruption, it's also a moment of opportunity, and we're good at seizing opportunity. So, AI and ML

00:45:12.899 - 00:45:29.570

are an outgrowth of our work. Um, we need to know more about some of their processes, models, and frameworks, um, and we can upscale on that. Um, we also need to know where they're gonna make our reference services better and more efficient and where they're not,

00:45:30.189 - 00:45:48.270

um, where they're gonna present us with new opportunities for teaching different skills and different subject matters. Um, we need to look to our access and tech services, especially RTS departments, as our innovators and ask them what's happening. Um, what's happening in the basement? What have you

00:45:48.270 - 00:46:09.389

all been doing down there? Um, and, and, and learn from their, you know, systems implementation. Um, AI and ML should, um, reinvigorate our discussions of learner-centered space planning. So there are lots of different ways to go with new library spaces, but we come back to what

00:46:09.389 - 00:46:23.909

do learners need. Um, and AI and ML should inform how we're training our staff and how we're hiring new people. If, if your team doesn't have someone who's going to be the AI advocate, um, you know, maybe you have an opportunity in your next hire to

00:46:23.909 - 00:46:41.979

find the person that is. Here are a few discussion questions to take with you over the next two slides. Um, so, um, kind of the, we'll, we'll, we'll start with the radical question. Um, how do we get back to that 1960s, 191970s effort, that 1979 moment

00:46:42.189 - 00:46:59.939

to have a nationally coordinated legal information system, and can, can we leverage AI and ML as our moment to get back to that? Um, what existing platforms and consortial arrangements can we lever leverage? Um, can Nelco be, uh, you know, a staging ground for new developments

00:46:59.939 - 00:47:17.149

and things like that? How do we explain the limits of AI and ML, um, to our patrons and staff, so that they're not over reliant, and so they have good AI literacy. That's always been our job is to help people understand, um, how, how tools can

00:47:17.290 - 00:47:35.270

lead to their misinformation, as well as their information. So how, how do we do that? Um, how can we train our patrons and staff to be sophisticated AI users? Um, how are we gonna staff in the coming decades? How do we secure resources for our own

00:47:35.270 - 00:47:50.899

development of AI tools? I mean, Cash shared that wonderful project, um, on, um, the Jim Crow laws earlier. So how do we get funding to be a part of that or even to lead those sorts of efforts? And then if our spaces are going to change,

00:47:51.100 - 00:48:06.610

um, how do we advocate for what, um, we and our patrons really need in the future? So those are some questions that we offer in our chapter to, to take with you. And that's it for our formal presentation, and, and, and now we can uh take

00:48:06.610 - 00:48:23.300

some time to answer your questions. Both so much. That was terrific. Um, I will start with a question that we have from the audience. I think it actually nicely kind of dovetails with some of those closing questions that you just left us with, especially around educating

00:48:23.300 - 00:48:41.090

our, our, our colleagues and users about strengths and weaknesses. So the question is, certain plagiarism detective detectors have a tendency to mistakenly flag original writings of ESL, English as a Second Language, and neurodivergent students as AI generated. What are your thoughts on how instructional law librarians

00:48:41.090 - 00:48:58.639

or law professors can address rising plagiarism without throwing vulnerable students under the bus? Do you wanna take that first, Sarah or do you want me to? Um, Either one cast, do you want it or do you want me to, I, I have one thought. Do you,

00:48:58.760 - 00:49:14.520

do you want, OK, all right, so are you sure, um, one of the things that we can do is always re-engage our students. So for instance, um, have students run the plagiarism reports, right, and have them comment on them, like have them engage with them. Um,

00:49:14.600 - 00:49:26.889

a second thing that I do is I always dig into the report and see what it's highlighting. Um, I think most of us have thresholds for the plagiarism, so And, and I, you know, I hate to admit this, maybe it's a dirty little secret, but if

00:49:26.889 - 00:49:39.790

it comes back at 5%, I'm not going to dig in. But if it comes back, you know, 35%, then I might start to read. Um, but I always go into the report itself, because early at UNC I got kind of burned in the way that the

00:49:39.790 - 00:49:55.320

questioner is, is saying, um, by not digging into the reports and, and seeing what they were flagging. Cass, do you come, do you use these reports too? And how do you deal with them? I, I don't, uh, partially because this is a problem with them and

00:49:55.320 - 00:50:13.179

partially because the product I personally have students produce doesn't affect me. What I do is I, I try to reconsider my assignments. So when this became a problem, I really thought about my assignments. in a way to figure out whether the goal of the assignment is

00:50:13.520 - 00:50:30.449

undercut by generative artificial intelligence, right? So I have a class, AI and big data and law practice, where technically there are several times they could use generative AI. To do some of the reports that they have to do, whether it's their data and ethics report or

00:50:30.449 - 00:50:43.449

things like that, but because so much of it is requiring this kind of critical thought, and I've built my rubric to really have them engage with their particular application of it and things like that, to get a generative AI system to produce an answer that would

00:50:43.449 - 00:50:59.669

meet the rubric would require such high level engagement with the generative AI system that I say go for it. I did have a student on our final reflection, 100% use generative AI on this reflection paper that was designed to summarize the threads and themes they saw

00:50:59.669 - 00:51:13.699

in student AI projects and report back on like what challenges they saw everyone having and what the report was this report on just generally what's wrong with AI, and I was like, The student did not even engage with the answer sufficiently to realize it didn't, it

00:51:13.699 - 00:51:29.949

was non-responsive, right? So I think the question becomes what things can you intercede for people who have iterative processes like writing, you can create moments where you have them do something, you give them feedback, and you're following the track through time in some fashion, whether that's

00:51:29.949 - 00:51:48.760

on Google Drive or something else, because if they copy and paste an entire page of thing, right, that's a whole different, you're like, hm. This just showed up in an hour versus, you know, if you're watching this kind of, there's ways to build in anti-generative AI,

00:51:48.979 - 00:52:08.169

the writing itself is so fundamental, but I think we have to just approach our assignments and really question what the goals are and whether generative AI necessarily undercuts our goals. Um, I'm gonna follow up on that with, uh, uh, moderators, uh, privilege. I'm gonna bring on

00:52:08.169 - 00:52:22.790

my own questions. I had the, the benefit of looking at a draft of your chapter ahead of time. So I saw your, your questions, uh, the questions that you leave the reader with at the end. And I was really struck with the point about Reinvigorating those

00:52:22.790 - 00:52:40.310

efforts from a couple generations ago to build nationwide legal information systems. And when I thought about that, it felt utopian in a specific way and that it would be kind of for, for the people from a, a, a, in a way that's not necessarily dominated by

00:52:40.310 - 00:52:56.350

specific vendors that doesn't fall into some of the uh the data cartel problems that we have. So I'm wondering about, given the I think for a lot of us in law libraries, when we're using generative AI with our patrons, uh, we are using it subject to

00:52:56.350 - 00:53:15.629

the parameters and the black box of the vendor who provides the technology. And so I'm wondering if you have a vision for using machine learning and artificial intelligence to democratize legal access in a way that partners with. Private industry that, uh, escapes it? Do we upskill

00:53:15.629 - 00:53:34.169

to the point that we are able to act, uh, as academics, uh, or, or as library professionals, uh, apart from big business or or if you have thoughts on the intersection of big business and this kind of utopian moment. All right, I'm going to jump on

00:53:34.169 - 00:53:45.969

it because Sarah's taking too long and I'm, I'm really excited. OK, so I think there's something to be said for these small movements that can create a foundation. So like on the Books teaches Mellon Foundation funding to create machine readable data sets of law. That's huge.

00:53:46.070 - 00:54:00.889

It's just creating text data sets, right, of these things to use for multiple projects is huge. But also, The movement with AI being so popular, especially at law schools, we have this way to leverage that for a good. So I'm in conversations with one of our

00:54:00.889 - 00:54:16.784

judges who's also part of the AI steering committee to develop a class, you know, I'm gonna get scooped at some point. I'm OK with that, but to create a class where students utilize generative AI to fast draft, right, to start. Helping them analyze, go through, and

00:54:16.784 - 00:54:33.304

draft and create towards the end because it's a semester, the documentation for courts that they would like to assist pro se patrons. One of the big ones is protective orders. Arizona protective orders are a nightmare, right? Like the, the process, the procedures, the laws involved, everything,

00:54:33.385 - 00:54:45.889

and we threw it all in Notebook LM and we started to play with it. And so the idea being that now we can use this class and this movement for students to be enhanced, you know, and engaging with AI and to use these tools to potentially

00:54:45.889 - 00:55:04.860

create work products, um, and, and so we tend to be kind of creative about these different facets of it and also start to think about ways that we can, um, create tools for others to innovate and create things to help access. In the way that like

00:55:04.860 - 00:55:23.520

games are created to be modded. Machine learning, like creating the infrastructure can allow us to create government in a way that we can have people mod to increase access and openness, Sarah. Oh, that's so cool. That's so cool. Um, yeah, I'll throw out a couple of

00:55:23.520 - 00:55:42.969

scattered ideas, you know, I, I got the chance to, um, work with Open Law Library, uh, to have some conversations with them a couple of years ago, and they're trying to standardize how legislatures, um, draft their laws. Um, and one of the, one of the things

00:55:42.969 - 00:56:02.780

that I think would be interesting is, you know, if, if we could get legislatures to, um, just Uh, write very or produce very, um, ingestible versions of their laws, like stop working in PDFs for instance, and, and that's not my idea, that's OLL's idea, um, but

00:56:02.780 - 00:56:20.679

that's one way that we could, um, Assist with this, and, and, and it would run around some of the vendors. I mean, some states have contracted out their official um online publishing to vendors, whereas others publish their laws, um, on their own sites, um, but the

00:56:20.679 - 00:56:41.020

ones who've contracted out, this would circumvent them. So that's, that's one opportunity. I think another opportunity is to, um, And maybe get our, you know, faculty involved in this, get courts considering, um, public hearings as um the truly public opportunities that they are, and maybe do

00:56:41.020 - 00:56:58.110

AI transcription of what's happening in court hearings, and then, you know, releasing that through court websites so that we have repositories of this, this real-time data coming out of courts, um, for interest, for instance, it's been occurring to me, um, A lot lately that I might

00:56:58.110 - 00:57:13.985

like to know what's going on in immigration hearings. Um, and that I really wish that I could just, just hear and have a big data set of what's going on in front of, um, judges hearing asylum cases these days. Um, and so, I think there, there

00:57:13.985 - 00:57:34.110

may be some unique opportunities in legislative publishing, in court hearings, um, that we could get involved in, that would circumvent, um, the big vendors, um, and, and, and be more egalitarian and more, um, public. Both, uh, I just want to flag a comment that came into

00:57:34.110 - 00:57:51.080

the Q&A while you were speaking that I think Mm Is conciliate. Uh, I just want to offer my opinion that AI won't help democratize access unless access to the underlying data is democratized. So I think, yeah, starting, starting with the source, uh, certainly makes sense. Um,

00:57:51.729 - 00:58:07.010

uh, I'm gonna throw it open to our audience. Uh, last, last moments for, for questions. Um, But since we are very close on time, I think we will probably not get any. So I will, uh, just take this opportunity to thank you both so, so much

00:58:07.010 - 00:58:23.334

for joining us today. Uh, this was fascinating. We are all looking forward to your chapter. Um, and, uh, I think, uh, that we did put the link to your publisher site in the Q&A. Thank you, Cass. So, um, We'll also have that linked on our repository

00:58:23.334 - 00:58:38.504

when the recording of this session goes there so that everyone can pre-order. So, uh, thank you both so much and I hope everyone has a fantastic rest of your afternoon, um, and I hope you'll join us again for another Biddle Speaker series presentation later this spring.

00:58:38.864 - 00:58:41.044

All right, bye everybody. Thanks everyone.