Andy White Anthropology
  • Home
  • Research Interests
    • Complexity Science
    • Prehistoric Social Networks
    • Eastern Woodlands Prehistory
    • Ancient Giants
  • Blog
  • Work in Progress
    • The Kirk Project >
      • Kirk 3D Models list
      • Kirk 3D Models embedded
      • Kirk 2D images >
        • Indiana
        • Kentucky
        • Michigan
        • Ontario
      • Kirk Project Datasets
    • Computational Modeling >
      • FN3D_V3
    • Radiocarbon Compilation
    • Fake Hercules Swords
    • Wild Carolina >
      • Plants >
        • Mosses
        • Ferns
        • Conifers
        • Flowering Plants >
          • Grasses
          • Trees
          • Other Flowering Plants
      • Animals >
        • Birds
        • Mammals
        • Crustaceans
        • Insects
        • Arachnids
        • Millipedes and Centipedes
        • Reptiles and Amphibians
      • Fungi
  • Annotated Publications
    • Journal Articles
    • Technical Reports
    • Doctoral Dissertation
  • Bibliography
  • Data

Kirk Project: Workflow from Point to 3D Morphometric Data

5/6/2016

2 Comments

 
I've spent most of today and yesterday trying to find a workflow path that gets me from the collection of 3D data (which I've been doing for months now) to a set of numbers that I can do meaningful analysis on.  This has involved, mostly, a lot of trial and error with various software programs. I think I've finally found a series of reasonable steps that will let me go from the 3D models to characterizations of shape data that I can examine visually and statistically.

This is post is mostly a way to document these steps for my own benefit, so that I can refer back to them in a week or a month when I've forgotten what I just figured out. This may not be the final procedure that I end up using for a large scale analysis. I'm putting this out there in public in case anyone finds it useful. Or in case I get hit by a bus and someone else needs to carry on with this important work.
Acquiring 3D Data
PictureScan in progress.
I'm using a NextEngine Desktop 3D scanner (UltraHD, Model 2020i). I'm scanning each point in two orientations: (A) one that exposes all the edges and (B) one that exposes the faces. Before scanning, I use a red pen (supplied with the scanner) to place a series of small dots along the edges of the point. These dots are used to align scan sets A and B into a single model (three points are required for alignment), so they should be placed in locations likely to show up in both scans.

I'm using the autodrive (rather than the multidrive) to rotate the points as they are scanned. I use part of an eraser to elevate the point off of the platform, and the padded to gripper to hold it firmly in place without damaging it.

For each scan set, I've got the scanner set to rotate the point through 10 divisions at the middle HD setting (67k points/square inch).  This may be overkill, but it's been working so far and I've been producing models far faster than I've been processing them (I've scanned about 90 points but have only processed the models for about 40). I scan with texture capture on, because I need to be able to see the red dots to do the alignment.

Processing 3D Data
PictureExample of raw scan data.
I keep copies of the raw (unprocessed) scan data in an archive folder. At the settings I'm using, the raw data for a set of scans from a single point occupies about 300-450 MB. I copy the files for each point and put them in a "processing folder."

​Processing is done in the ScanStudio software that came with scanner. The first thing I do is trim away the non-point things that were captured in the scan. This is a simple operation: you just select what you don't want and delete it. Because I'm scanning in two different orientations, it doesn't matter if a small bit of the actual point gets deleted where it's touching the eraser and gripper - data for those areas will be contained in the other set of scans.

PictureAlignment in process.
After the two sets of scans are trimmed, I align A and B. This is done by determining which of my red dots I can most reliably locate on both scan sets and marking them with color-coded pins. Again, this is a fairly simple process. I've found that some care in this step can significantly reduce the amount of time you have spend cleaning up the model later. 

After at least three pins are placed (I'm not sure that placing more than three actually leads to better results), you hit the "align" button and see what you get. I find it helpful to switch the display mode to "solid" at this point (removing the photo-like surface texture) because it is much easier to see how the two scan sets actually lined up. If the alignment is bad, you can tell. Sometimes an alignment issue can be mitigated by just going back and re-setting the pins and doing the alignment again. Other times it is apparent that the best solution is to go back to square one and re-generate the raw scan data (i.e., put the thing on the scanner and start over).

PictureFused model.
Assuming the alignment is okay, I fuse the two scan sets together with the "fuse" command. This is a one-button operation.  This is another one button operation. I'm using these fuse settings:
  • Volume Merge,
  • Resolution Ratio 0.5 
  • "Create Watertight Model" unchecked
  • "Include Textures" checked
  • Texture Blending 5 pixels
Fusing scan sets A and B produces a new mesh, conveniently labeled "C." If I'm satisfied with the way C looks, I go ahead and delete A and B at this point.

The fused model requires some clean up. There are often small bits "floating" around the edges and at locations on the surface were data from A and B overlapped. These can be removed using the trim tool. The cleaner the model, the less time this will take, obviously. I really feel like the scanner has good days and bad days: sometimes it seems like the scans are really messy no matter what you do, and sometimes things just work out easily. Regardless, I'm not sweating pixel-level details on these models because it won't affect the kind of morphometric analysis I'm planning on doing. I just need the models to be fairly good approximations of the actual point.

After cleaning up the fused model, I run the "remesh" operation (resolution ratio = 0.9) to fill the holes and even things out a bit. Sometimes this reveals a few more defects that can be addressed through limited trimming. In that case, I'll run the "fill holes" operation ("smooth" and "smooth boundaries" checked) afterward to fill any remaining holes in the mesh.

The file I'm working with to produce these images is about 23 MB at this point. I use the "simplify" tool (set at 0.0050") to reduce the file size to about 5 MB. Then I create a .STL file for exporting the model to Sketchfab and a .PLY file to use for analysis in Landmark.  Here is the model on Sketchfab:

Kirk 5984 by aawhite on Sketchfab

Deriving 3D Landmark Coordinates
This is the part of the equation I just figured out the other day. I discovered the Landmark software package (available here) that allows you position sets of landmarks on the surfaces of 3D models. The software was designed with three-dimensional, irregular biological structures (such as bones) in mind, but will work great for projectile points as well. It is mainly a software to acquire, rather than analyze, coordinate data (at least in my plan). 

Even though the manual says the software will work with .STL files, it didn't seem to want to do it for me. So I converted a batch of 10 Kirk models into .PLY files so I could import them into Landmark and work my way through the process of generating data. I added the model I just created above to the batch to illustrate the steps in this software.
Picture
For a trial run, I defined five replicable landmarks that can be placed in the haft region when the point is orientated with the tip pointing up: the points of greatest constriction (s0 and s1), the points of greatest width below the notches (s2 and s3), and the point of greatest divergence from horizontal along the basal edge (s5). The labels are automatically applied to the points as you place them.

As in other software like this, you place the landmarks in the same order each time because the end result is to going to be a file of numbers with XYZ coordinates corresponding to the locations of those landmarks on each of the objects in your assemblage. The next software that reads that file is going to assume that the coordinates are all in the same order, so they better be or you'll get nonsense back. Landmark helps with this by making the process of applying consistent sets of landmarks to different objects semi-automated.

Here is a close up showing the landmarks placed on the surface of the model. You can use your mouse to drag them around and get them exactly where you want them:


Picture
After I placed the five landmarks on a group of ten Kirk point models, I exported the coordinates of the landmarks by highlighting all the models (listed in the pane on the left) and using the "export" function (in the Project menu). I had to play around with several different options to get a file that would eventually work in the analysis software I found (see below). Supposedly I should be able to export files in a format that can be read directly by MorphoJ, but I couldn't get it to work. I had to export the data in the .DTA format and then just edit the text file myself to a format that MorphoJ could read. The .DTA format has the advantage of being a single file with all the coordinates clearly organized, so editing it was no big deal.

Analysis of 3D Coordinate Data 

Once you have 3D coordinate data, what do you do with them? I was surprised to find that my most beloved data analysis package, JMP, doesn't seem to be able to hand 3D coordinate data. Some of software that can do tricks with 3D data, like EVAN, costs money (which I'd like to avoid spending on a product that I'm not sure is actually going to do what I want it to do).  I downloaded various free software packages and played around with several of them. The one I finally got work is called MorphoJ (available here).

The first trick is getting the 3D coordinate data exported from Landmark to be read properly in MorphoJ. As I said above, this wasn't as easy as advertised (here's the relevant section of the MorphoJ manual). Eventually I gave up on the "easy" route and just edited the data from Landmark into a standard comma-delimited text file using Notepad. For future reference, the input data file should look like this:
Picture
The first line gives MorphoJ the column labels. Each line after that contains data for a single Kirk point: the label of the point, the XYZ coordinates of the first landmark, the XYZ coordinates of the second landmark, etc. Don't put commas at the end of lines, don't put in any tabs, etc.

After you create the data file, you import it into MorphoJ using File-->Create New Project. In the dialog that comes up, click "3 dimensions," select "text" as file type, and then navigate to the text file with the coordinate data. If there are no problems with the import it will tell you so. 

Once the data are in there, they'll show up on the Project Tree. Select the data set, go to the Preliminaries menu, and choose "New Procrustes Fit." This performs a Procrustes analysis that rotates, translates, and scales the objects in space to minimize their differences:
Picture
The graphic above shows the results of the Procrustes analysis, with the numbered points representing the centroids and clouds of smaller points around them representing the spread of actual data values. Notice that the points are flipped vertically and horizontally from the way I showed them in the images above, and also that s0 is now "1," s1 is now "2," etc. The results can be viewed along three different axes.

The Procrustes data can be used to do a Principal Components Analysis, which reduces the three axes down to two.  Here is what that looks like:
Picture
Now we're getting somewhere!  The last hurdle will be to figure out if/how I can export the raw principal component data so I can analyze it in another software package.

Working my way through these steps was a "proof of concept" exercise that I needed to do before scaling the analysis up to the full sample. I've been down a lot of dead ends with software.  I'm hoping this is the combination that gets me to a full-scale analysis. The five landmarks I used for the test run really don't take advantage of the 3D model data that I have, so I'll need to start thinking in three dimensions rather than two. And I still don't know how to make use of the data from curves. Once I get those things sorted out, it will be really interesting to see how variation patterns out with a much larger sample (I'm aiming for 100) from a single county and a single raw material. Based on what I've seen as I've been creating the 3D models, I will be surprised if some component of temporal variation is not detectable.  

And then, of course, I'll need to do about 10,000 more of these to see what's going on in the rest of the country.
2 Comments

Data from "Functional and Stylistic Variability in Paleoindian and Early Archaic Projectile Points from Midcontinental North America"

3/3/2015

1 Comment

 
Picture
I have added an Excel file of the basic dataset for my 2013 paper in North American Archaeologist to the "Data" section of the website.  The file contains basic information on provenience (county), UTMs (county center), typological category, and morphometric data for the 1,771 Paleoindian and Early Archaic projectile points that I used in that study.  Like the 2014 AENA paper, the NAA paper was produced from a portion of the analysis in my dissertation. 

The samples I used for the morphometric analysis in my dissertation and in the NAA paper were identical, so the data in the Excel file are also in the appendices of my dissertation.  I'm hoping that providing the data in an electronic format will save someone a great deal of time doing data entry, and will encourage the use of the dataset that took me who-in-the-hell-knows-how-many hours and miles to collect, compile, and produce.  The measurements used, as well as the procedures for taking them, are defined in the paper and in my dissertation.

The ultimate goal of the two analyses (raw material and morphometric) was to produce a quantitative description of the apparent sequence of material culture change from homogenous (Early Paleoindian) --> regionalized (Late Paleoindian) --> homogenous (Early Archaic) that characterizes the Pleistocene-Holocene transition in the Midcontinent.  A quantitative description allowed an "apples to apples" comparison with data from model experiments, providing a basis for evaluating some alternative scenarios explaining the regionalization as a result of various changes in social network structure.  As the time to my defense was ticking away, I had to sacrifice some of the modeling work in order to get finished.  I was able to draw some conclusions, but a satisfying analysis of the "social boundary" question is still in the future.  Once I get set up at my new job I'll be able to restart the modeling work, add data from the southeast to my dataset, and reboot on the question of the social networks of early hunter-gatherers in eastern North America.


1 Comment

    All views expressed in my blog posts are my own. The views of those that comment are their own. That's how it works.

    I reserve the right to take down comments that I deem to be defamatory or harassing. 

    Andy White

    Follow me on Twitter: @Andrew_A_White

    Email me: andy.white.zpm@gmail.com

    Enter your email address:

    Delivered by FeedBurner


    Picture

    Sick of the woo?  Want to help keep honest and open dialogue about pseudo-archaeology on the internet? Please consider contributing to Woo War Two.
    Picture

    Follow updates on posts related to giants on the Modern Mythology of Giants page on Facebook.

    Archives

    January 2023
    January 2022
    November 2021
    September 2021
    August 2021
    March 2021
    June 2020
    April 2020
    March 2020
    January 2020
    December 2019
    November 2019
    October 2019
    September 2019
    May 2019
    April 2019
    January 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    May 2017
    April 2017
    March 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016
    March 2016
    February 2016
    January 2016
    December 2015
    November 2015
    October 2015
    September 2015
    August 2015
    July 2015
    June 2015
    May 2015
    April 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    September 2014
    August 2014
    June 2014
    May 2014
    April 2014
    March 2014

    Categories

    All
    3D Models
    AAA
    Adena
    Afrocentrism
    Agent Based Modeling
    Agent-based Modeling
    Aircraft
    Alabama
    Aliens
    Ancient Artifact Preservation Society
    Androgynous Fish Gods
    ANTH 227
    ANTH 291
    ANTH 322
    Anthropology History
    Anunnaki
    Appalachia
    Archaeology
    Ardipithecus
    Art
    Atlantis
    Australia
    Australopithecines
    Aviation History
    Bigfoot
    Birds
    Boas
    Book Of Mormon
    Broad River Archaeological Field School
    Bronze Age
    Caribou
    Carolina Bays
    Ceramics
    China
    Clovis
    Complexity
    Copper Culture
    Cotton Mather
    COVID-19
    Creationism
    Croatia
    Crow
    Demography
    Denisovans
    Diffusionism
    DINAA
    Dinosaurs
    Dirt Dance Floor
    Double Rows Of Teeth
    Dragonflies
    Early Archaic
    Early Woodland
    Earthworks
    Eastern Woodlands
    Eastern Woodlands Household Archaeology Data Project
    Education
    Egypt
    Europe
    Evolution
    Ewhadp
    Fake Hercules Swords
    Fetal Head Molding
    Field School
    Film
    Florida
    Forbidden Archaeology
    Forbidden History
    Four Field Anthropology
    Four-field Anthropology
    France
    Genetics
    Genus Homo
    Geology
    Geometry
    Geophysics
    Georgia
    Giants
    Giants Of Olden Times
    Gigantism
    Gigantopithecus
    Graham Hancock
    Grand Valley State
    Great Lakes
    Hollow Earth
    Homo Erectus
    Hunter Gatherers
    Hunter-gatherers
    Illinois
    India
    Indiana
    Indonesia
    Iowa
    Iraq
    Israel
    Jim Vieira
    Jobs
    Kensington Rune Stone
    Kentucky
    Kirk Project
    Late Archaic
    Lemuria
    Lithic Raw Materials
    Lithics
    Lizard Man
    Lomekwi
    Lost Continents
    Mack
    Mammoths
    Mastodons
    Maya
    Megafauna
    Megaliths
    Mesolithic
    Michigan
    Middle Archaic
    Middle Pleistocene
    Middle Woodland
    Midwest
    Minnesota
    Mississippi
    Mississippian
    Missouri
    Modeling
    Morphometric
    Mound Builder Myth
    Mu
    Music
    Nazis
    Neandertals
    Near East
    Nephilim
    Nevada
    New Mexico
    Newspapers
    New York
    North Carolina
    Oahspe
    Oak Island
    Obstetrics
    Ohio
    Ohio Valley
    Oldowan
    Olmec
    Open Data
    Paleoindian
    Paleolithic
    Pilumgate
    Pleistocene
    Pliocene
    Pre Clovis
    Pre-Clovis
    Prehistoric Families
    Pseudo Science
    Pseudo-science
    Radiocarbon
    Reality Check
    Rome
    Russia
    SAA
    Sardinia
    SCIAA
    Science
    Scientific Racism
    Sculpture
    SEAC
    Search For The Lost Giants
    Sexual Dimorphism
    Sitchin
    Social Complexity
    Social Networks
    Solutrean Hypothesis
    South Africa
    South America
    South Carolina
    Southeast
    Stone Holes
    Subsistence
    Swordgate
    Teaching
    Technology
    Teeth
    Television
    Tennessee
    Texas
    Topper
    Travel
    Travel Diaries
    Vaccines
    Washington
    Whatzit
    White Supremacists
    Wisconsin
    Woo War Two
    World War I
    World War II
    Writing
    Younger Dryas

    RSS Feed

    Picture
Proudly powered by Weebly