Model Accuracy

Models were developed using Maxent version 3.3.3e. We used the cumulative probability output format. Although models were developed using data from outside of the Brandt boreal/hemiboreal boundary, our predictions were constrained to this region.

We ran the model 10 times using bootstrapped subsamples of the BAM/BBS dataset, each time holding out a randomly-selected 50% of occurrence locations as a test dataset for internal model validation. Model predictions were averaged across the 10 bootstrap replicates. The AUC value can be interpreted as the likelihood that a randomly-selected presence location will have a higher suitability score than a randomly-selected background location. 

Model accuracies were assessed by calculating the area under the curve (AUC) of the receiver operating characteristic (ROC) plot (Fielding & Bell, 1997 ) for each of the 10 test data sets. AUC values were also averaged across the replicates.

In general, models were reasonably accurate in their prediction of species' distributions. Average AUC scores ranged from 0.56 for American Robin to 0.97 for American Tree Sparrow (see below). Across all 96 species, AUC scores averaged 0.81 ±0.09 (SD). Models for 30 species were considered acceptable 0.7 ≤ AUC < 0.8), 36 were excellent (0.8 ≤ AUC < 0.9), and 18 had outstanding discrimination ability (AUC ≥ 0.9) (Hosmer & Lemeshow, 1989 ). AUC scores reflected the ability to discriminate among different levels of habitat suitability within the greater boreal and hemiboreal region. Thus, species with distinct range limits within this region were more accurately predicted.

 

Maxent model accuracy (area under the curve, AUC) and number of occurrence records (n) by species. AUC = mean area under the curve across 10 bootstrapped Maxent model runs. For each bootstrap replication, 50% of species' occurrences were used to train the model, and the remaining 50% were used to test the model.

Species AUC n

ALFL

0.69

10052

AMCR

0.67

15176

AMGO

0.73

11591

AMRE

0.72

9371

AMRO

0.56

22016

ATSP

0.97

522

BAOR

0.85

4474

BAWW

0.76

7170

BBMA

0.92

2407

BBWA

0.88

2182

BCCH

0.67

11945

BHCO

0.78

8050

BHVI

0.79

5731

BLBW

0.84

4002

BLJA

0.75

9484

BLPW

0.92

1278

BOCH

0.85

2225

BRCR

0.81

2046

BTBW

0.89

2994

BTNW

0.81

5914

CAWA

0.84

2169

CCSP

0.89

4424

CEDW

0.67

9781

CHSP

0.64

15108

CMWA

0.86

1392

COGR

0.79

8629

CONW

0.94

1050

CORA

0.68

9801

CORE

0.96

751

COYE

0.66

12978

CSWA

0.80

7373

DEJU

0.76

8801

Species AUC n

EAKI

0.82

5220

EAPH

0.81

3833

EAWP

0.83

3957

EVGR

0.80

2589

FOSP

0.91

2057

GCFL

0.85

4482

GCKI

0.79

5557

GCTH

0.96

469

GRAJ

0.84

4535

GRCA

0.80

4764

HAFL

0.94

1619

HETH

0.70

10989

HOWR

0.83

5570

LCSP

0.92

1653

LEFL

0.69

9163

LISP

0.81

4707

MAWA

0.77

9244

MAWR

0.88

792

MGWA

0.94

1637

MODO

0.77

9455

MOWA

0.80

5166

NAWA

0.79

8367

NOWA

0.72

5101

OCWA

0.87

3799

OSFL

0.77

2797

OVEN

0.74

12047

PAWA

0.88

1914

PHVI

0.86

1994

PIGR

0.89

605

PISI

0.81

4982

PIWA

0.90

1278

PUFI

0.77

4219

Species AUC n

RBGR

0.78

6315

RBNU

0.74

7895

RCKI

0.77

8655

RECR

0.88

1094

REVI

0.66

16397

RUBL

0.87

544

RUGR

0.74

3073

RWBL

0.74

11824

SAVS

0.75

10184

SCTA

0.88

1889

SEWR

0.93

1135

SOSP

0.69

14573

SWSP

0.75

4946

SWTH

0.74

11654

TEWA

0.85

4701

TOSO

0.95

567

TOWA

0.95

1472

VATH

0.93

1987

VEER

0.79

7101

VESP

0.86

4857

WAVI

0.77

7561

WBNU

0.82

2307

WCSP

0.93

1784

WETA

0.92

2365

WEWP

0.91

1739

WIWA

0.84

3166

WIWR

0.77

7677

WTSP

0.71

14221

WWCR

0.84

2691

YBFL

0.85

3365

YRWA

0.70

12392

YWAR

0.67

11661