Back in the late 2000's, I took the terrifying step of creating folders on my computer to start pursing my formal dissertation research. Around the same time, I realized that my system for organizing my paper files had become a sandbag. The physical compartments I was using to segregate "different" aspects of my work were hurting my ability to see and explore the overlapping areas of several inter-connected problems. I tore everything apart and put it back together again so the overall structure was different, the grains of information were different, and the "bins" were collapsed into a single well that I could draw from. In order to stop blindly analyzing the different parts of the elephant and start trying to understand the whole animal, you first have to understand  that you're looking at pieces of a much larger puzzle.

It was more of a strategy than an epiphany. 

Last week I got into the nitty-gritty of a SEAC paper I'm writing with David G. Anderson (University of Tennessee). We're using various large datasets to try to describe and interpret patterns of change in archaeological remains that could be related to changes in the size, structure, and distribution of human populations in the Eastern Woodland during the Late Pleistocene and Early Holocene.  

As I started pulling together information (from PIDBA, DINAA, and my ongoing radiocarbon compilation) and thinking about how to organize it, I realized that keeping the databases separate was both a logistical hassle and an analytical problem. I invested in dumping all the information into a single relational database that we can use for this paper and that I'll continue to update in the future. I've been calling it "Megabase" in my head. So that's what it is until it gets a better name.

​Here is an illustration that I'll briefly discuss:

On the left is what I'm building now. I used GIS to generate a listing of "center" UTM coordinates (n=2097) for every county in the eastern US (everything east of the first tier of states west of the Mississippi River) and much of eastern Canada. I'm calling that the "County Core." That coordinate list lets me easily create a spatially-reference file for whatever other information I want from any of the other databases without needing to know the exact locations of archaeological sites.  Making a county-level map of all eastern radiocarbon dates in the database (9,533 and counting) in the eastern US is just a matter of a few button clicks in Access, Excel, and GIS. The same is true of the PIDBA data, the Kirk Project data, the household archaeology data, and the DINAA data. 

The Megabase of Today will be fine for the SEAC paper and for the near future. It will be able to do a lot. Ideally, however, the Megabase of the Future will have DINAA serving as both a "router" for data that is attached to a Smithsonian Trinomial and an analytical tool in its own right. One issue is that not all states are currently participating (and therefore not all Smithsonian Trinomials -- the "addresses" for sites -- are in the system).  Another issue is that the site forms (and therefore the site information that is collected and stored) differ by state. To reach its full potential, DINAA data will have to be supplemented by additional data about the materials recovered from sites, how sites were recorded, etc. Ensuring that we're making "apples to apples" comparisons will be a significant chore -- DINAA currently has information on somewhere in the neighborhood of half a million sites. You can't just sit on your couch and cross-check all that.

I know enough to be dangerous with a computer, but I'm not sufficiently sophisticated to know the nuts-and-bolts options for building the Megabase of the Future. In 2015 we did a sort of "proof" of concept to demonstrate that the EWHADP and DINAA could be linked together. I'm not sure if that is they way to go or not. Perhaps there's something that can be done with blockchain technology -- it sure sounds cool.

Anyway, I'm going to get the Megabase of Today functional in time to do the analysis for the SEAC paper we'll give in a month. If you're interested in talking about the Megabase of the Future, please let me know.

Readers of this blog from the pre-Swordgate era may remember the Eastern Woodlands Household Archaeology Data Project (EWHADP) that I initiated in February of 2014. The goal of the EWHADP is to assemble and make available information about prehistoric residential structures in eastern North America. The project has it's own website where you can read all about it and access the data.

After an initial startup pulse, I had to put the project on hold when I started teaching at Grand Valley (Fall of 2014). Emily Gilhooly, one of my undergraduate students at Grand Valley, worked on the project during the winter of 2015 and began the long process of updating the database by checking and re-coding every single entry. As I prepared to make the move to the University of South Carolina in the spring of 2015, I started a GoFundMe campaign to raise some cash to pay a research assistant to work on the project. Thanks to the generosity of several donors, that campaign was successful. With the hiring of USC graduate student Laura Clifford, the project was up and running again in the fall of 2015. While Laura made a lot of headway tracking down references and moving the database forward, however, she soon found a better long-term employment situation. So I hit the pause button again on the EWHADP.

The project stayed paused as I moved from the SCIAA building into my new lab space, worked my way through my first two years of teaching at South Carolina, and initiated some excavation fieldwork at a site on the Broad River. After that field school, I hired one of my students (Sam McDorman) to do the bulk of the basic laboratory processing of the materials we recovered. With the artifacts from 38FA608 washed and cataloged, I can start analysis. And I can move Sam on to another project: the EWHADP.

Of the $3400 in donated EWHADP funds that I came with, there was about $2870 left at the beginning of the semester.  That money should be sufficient to get through at least 2.5 of the three goals I have:

1) assemble primary references for all information in the database;
2) check and re-code existing information in the database, supplying missing information and adding greater detail;
3) add new information to the database.

I don't expect to get through these goals quickly, but the wheels are now in motion again. The EWHADP is staffed, funded, and exists in a dedicated office space with room for files, books, piles of stuff, a scanner, and a computer. When the second goal is completed, you'll get an updated database that should be an order of magnitude better than the one that exists now. And then you'll start getting updates to the website as we begin adding in new information.

I'd like to again thank those that donated to this project and haven't questioned why it has periodically slowed down over the years: anyone who juggles knows that it's difficult to keep everything up in the air at all times. Your patience is appreciated. Thank you. And stay tuned.

We're now into the fourth week of the semester here at the University of South Carolina. As usual I've been writing for this blog less than I'd like (I have several unfinished draft posts and ideas for several more, and there's currently a backlog of Fake Hercules Swords). A good chunk of my time/energy is going into the Forbidden Archaeology class (you can follow along on the course website if you like -- I've been writing short synopses, and student-produced content will begin to appear a few weeks from now). Much of the remainder has gone into pushing forward the inter-locking components of my research agenda. This is a brief update about those pieces.

Small-Scale Archaeological Data

At the beginning of the summer I spent a little time in the field doing some preliminary excavation work at a site that contains (minimally) an intact Archaic component buried about 1.9 meters below the surface (see this quick summary).  Based on the general pattern here in the Carolina Piedmont and a couple of projectile points recovered from the slump at the base of the profile, my guess is that buried cultural zone dates to the Middle Archaic period (i.e., about 8000-5000 years ago).​

My daughter washed some of the artifacts from the site over the summer, and I've now got an undergraduate student working on finishing up the washing before moving on to cataloging and labeling. Once the lithics are labeled we'll be able to spread everything out and start fitting the quartz chipping debris back together. Because I piece-plotted the large majority of the lithic debris, fitting it back together will help us understand how the deposit was created. I'm hoping we can get some good insights into the very small-scale behaviors that created the lithic deposit (i.e.,perhaps the excavated portion of the deposit was created by just one or two people over the course of less than an hour).

When the archaeology faculty met to discuss the classes we'd be offering in the spring semester, I pitched the idea of running a one-day-per-week field school at the site. Assuming I can get sufficient enrollment numbers, that looks like it's going to happen. The site is within driving distance of Columbia, so we'll be commuting every Friday (leaving campus at 8:00 and returning by 4:00). The course will be listed as ANTH 322/722. It's sand, it's three dimensional, and it's pretty complicated -- it's going to be a fun excavation. I'll be looking to hire a graduate student to assist me on Fridays, and I'll be applying for grant monies to cover the costs of the field assistant's wages, transportation, and other costs associated with putting a crew in the field. 

Large-Scale Archaeological Data

Some parts of my quest to assemble several different large-scale datasets are creeping along, some are moving forward nicely, and some are still on pause.

Complex Systems Theory and Computer Modeling

Complex systems theory is what will make it possible to bridge the small and large scales of data that I'm collecting. Last year, I invested some effort into transferring my latest computer model (FN3_D_V3) into Repast Simphony and getting it working. I also started building a brand new, simpler model to look at equifinality issues associated with interpreting patterns of lithic transport (specifically to address the question of whether or not we can differentiate patterns of transport produced via group mobility, personal mobility between groups, and exchange).  

As it currently sits, the FN3_D_V3 model is mainly demographic, lacking a spatial component. Over the summer I used it to produce data relevant to understanding the minimum viable population (MVP) size of human groups. Those data, which I'm currently in the process of analyzing, suggest to me that the "magic number of 500" is probably much too large: I have yet to find evidence in my data that human populations limited to about 150 people are not demographically viable over spans of several hundred years even under constrained marriage rules. But I've just started the analysis, so we'll see. I submitted a paper on this topic years ago with a much cruder model and didn't have the stomach to attempt to use that model to address the reviewers' comments. I'm hoping to utilize much of the background and structure of that earlier paper and produce a new draft for submission quickly. I also plan to put the FN3_D_V3 code online here and at OpenABM.org once I get it cleaned up a bit. I also discuss this model in a paper in a new edited volume titled Uncertainty and Sensitivity Analysis in Archaeological Computational Modeling (edited by Marieka Brouwer Burg, Hans Peeters, and William Lovis). 

It will be a relatively simple thing to use the FN3_D_V3 model in its non-spatial configuration to produce new data relevant to the Middle Paleolithic mortality issue I discussed at the SAA meetings a couple of years ago. I'm also going to be working toward putting the guts of the demographic model into a spatial context. That's going to take some time.

American Antiquity is the flagship journal of archaeology in the United States. While I haven't yet published a research paper in it, I'm happy to say that I've got a book review in the current issue. Building the Past: Prehistoric Wooden Post Architecture in the Ohio Valley-Great Lakes is a collection of papers edited by Brian G. Redmond (Cleveland Museum of Natural History) and Robert A. Genheimer (Cincinnati Museum Center). The book contains a lot of useful data and some really interesting insights, and I enjoyed reading it.  I'll be going through the chapters again in detail when I finally get back around to working on the database for the Eastern Woodlands Household Archaeology Data Project.

Shortly after I built the website for the Eastern Woodlands Household Archaeology Data Project (EWHADP) in the spring of 2014, I talked about the project to a workshop sponsored by the Digital Index of North American Archaeology (DINAA) (there's a link to that presentation here and some short blog posts about how my work fits into DINAA here and here). In that 2014 presentation, delivered while I was unemployed, I offered an incentive program that I hoped would spur interest in the EWHADP: beer to the first person who cited the database. With an uncertain future, I scaled my offer to whatever my employment situation happened to be at the time of citation:

At the SAA meetings last week, I met Jayur Mehta and bought him a beer.  Mehta cited the EWHADP database in a paper about the Carson site, a Mississippian site in Coahoma County, Mississippi. If I remember correctly, the paper is currently under review, so I won't say anything else about it at this point. I'm glad he was able to make use of the database, and I look forward to reading the paper.

The alert reader will notice that Dr. Mehta is holding but a single plastic cup of beer in his hand, while my incentive program clearly shows that six beers will be awarded if I am employed in a non-tenure-track position, which I am.  I will point out that the cost of the beer he is holding, purchased in a Disney facility, actually exceeded the cost of a six-pack of PBR purchased in a normal part of the world (here is a link to $4.99 six pack of PBR on sale at Binny's Beverage Depot, the first place that came up when I Googled "PBR six pack cost").  So I think I fulfilled my pledge.

Mehta was unaware of my offer when he cited the database in his paper, suggesting that my incentive program was not behind his decision to use the EWHADP dataset.  Rather, it appears, he recognized the usefulness of the data for the project he was working on.  I hope that more people use the data. It's possible that someone else already has but I'm not aware of the citation - I'm not sure how I would ever know unless I see it or someone tells me about it.

The EWHADP has been largely dormant for a while now. It's stalled midway through an effort to re-code several of the key variable related to structure size and shape. I'm hoping to get the update done at some point this summer and get a newer, larger version online. I plan to use it myself for some research that will build on the 2013 JAA paper that spawned the original dataset.

If you cite the database, please let me know. I'll create a page for those citations on the EWHADP site. I'm interested to see how people use the data, and tracking and understanding use will help me to enlarge and improve the database.

I hope to be able to announce the latest iteration of the Eastern Woodlands Household Archaeology Data Project (EWHADP) database soon.  The last database release (containing information on over 2100 structures) was all the way back in March of 2014.  Gah!

​The project got off to a quick start in February of 2014, but stalled when I had to direct my energies elsewhere later that year. I spent the 2014-2015 academic year teaching a 4-4 at Grand Valley State University, and it was difficult to find time to do anything with the EWHADP other than teach look at the box of files and participate in a trial linking of the EWHADP database with DINAA.  GVSU undergraduate Emily Gilhooly did make some progress on the database, continuing the process of consulting the original records in order to re-code some fields and add new data to others.  

With donated cash in hand from a successful GoFundMe campaign, I was able to hire University of South Carolina doctoral student Laura Clifford to work on the project this semester.  Laura's first job is to finish the checking and re-coding of all the records currently in the database. She's working on that now.  When that task is complete, we'll make the new database available and she'll move on to the next job: adding new records.  That will involve tracking down leads from publications we've already seen as well as finding new sources of data in print publications and online. Eventually  I hope to give Laura the the keys to the EWHADP website so that she can write the "What's New" blurbs as she adds new structures, update the maps, and keep the online bibliography up to date. 

I would like to keep this project going next semester, but I'm not going to ask for more money until I have some results from this semester to demonstrate success.  

I would like to again thank those made a cash contribution to help get the EWHADP out of mothballs and running again: David Cusack, Ken Kosidlo, Josh Wells, and a donor who wishes to remain anonymous. In addition to my sincere gratitude, I have (as promised) a limited edition bumper sticker for each of you. And I owe you beers.

I just returned from several days at the DINAA (Digital Index of North American Archaeology) workshop that is happening at Indiana University South Bend this week.  The first two years of the DINAA project have focused on building a comprehensive, accessible database of archaeological sites in Eastern North America.  What the PIs and their core team (Josh Wells, David Anderson, Eric Kansa, Sarah Kansa, Steve Yerka, R. Carl DeMuth, Kelsey Noack Myers, and Thad Bissett) have accomplished to date is pretty remarkable:  in the pilot phase of the project, the team has assembled and made available primary data on over 340,000 recorded archaeological sites from ten states (with more data on the way).  This endeavor required navigating numerous technical, logistical, and political challenges.  What a great job they've done.  Bravo!

The DINAA project will benefit the archaeological community (and other constituencies) in a number of ways.  Some of those are obvious now, and some will become apparent as we become able to think in practical terms about the power and potential of a large, unified dataset that integrates and crosscuts the traditional (i.e., state-level) territories within which archaeological site data are managed. 

The benefit that I am most excited about is the potential of DINAA to act as a "bridge" among otherwise disconnected datasets. The key to this inter-linking is DINAA's granularity:  by making the archaeological site number the primary means by which information is organized, any datasets that reference a site number could be inter-linked through DINAA regardless of what primary information about the site is held by DINAA.  I'm compiling data on prehistoric house structures, for example, that could be linked, through DINAA, to other datasets using the key attribute of "site number."  Imagine being able to click on the site number associated with a structure in the Eastern Woodlands Household Archaeology Data Project (EWHADP) database and being led to a record in DINAA that provides links to a database of radiocarbon dates, or a spreadsheet of feature contents, or images from museum collections or field note archives, or bibliographic references for reports, dissertations, or academic papers that are also associated with that site number.  Or imagine being able to make a query for floral remains from Middle Woodland features within a 100 km radius of the site with that structure.  To say that that kind of inter-linking would be a powerful tool for research and scholarship is a great understatement.  As I argued in a presentation via Skype to the DINAA workshop that was held at the University of Tennessee in March, inter-linking of datasets would: (1) allow us to greatly expand the scale of questions we can address; (2) allow us to gather data for addressing those "big scale" questions much more efficiently; and (3) be a catalyst for developing and testing new interpretations of the past.  Engineering a system of links to connect diverse datasets would be a game changer.

At the workshop this week, we spent some time thinking about ways to actually accomplish such a linking.  The structure data I've been gathering is a good "test case" primarily because it spans the same area as the DINAA data and includes information from many sites (and is also open and freely accessible).  Information needs to flow both ways.  First, when a site record is called up in DINAA, it should be able to make a call to the EWHADP (and any other linked datasets) to see if there are records associated with that site number.  Second, site/structure records in the EWHADP should include a pointer to the appropriate record in DINAA.  The second part is simple: since the URLs associated with site records in DINAA are "stable," I can just put hyperlinks in my database that point to the DINAA record.  The second part is a little trickier.  Our first step was to put the current (as of March 2014) EWHADP database on GitHub (here) so that it would be open and accessible.  GitHub automatically tracks when changes are made to a file.  The next step (I think) will be to configure the GitHub page so that it sends a message to DINAA when the dataset changes.  This will allow the records in DINAA to be updated as new records are added to the EWHADP. 

It is the linking mechanisms that are important.  The EWHADP data do not become a "part" of DINAA but are simply referenced by DINAA.  I maintain control and responsibility for the EWHADP part of the equation.  This is important because compilation of that dataset is ongoing:  it's a database of records that I'm still collecting rather than something like a static set of measurements that relate to a single assemblage or site.  I hope others who are developing similar datasets think about how we might link them all together through DINAA.  The corpus of scholarship relevant to archaeological work in this part of the world is simply too large and diverse to live in a single place.  A distributed approach using a "bridge" such as DINAA to link datasets is going to be much more effective and useful than trying to house everything in one central place.

We all have a lot to gain by supporting the construction of this tool.  It is going to unleash the potential energy stored in the work we've already done and provide a "living" structure that will significantly increase our capacities to find, share, utilize, and build on archaeological information.  The DINAA project needs to move forward in a big way.