Welcome to Eva!

Eva is an open-source project intended for local use by HMIS Administrators in Continuums of Care (CoCs) around the U.S. and its territories. Eva is designed to help you (1) assess the accuracy and completeness of the data within your HMIS, and (2) understand your homeless response system’s flow and performance. Using Eva does not result in reporting or sharing data with HUD and use of Eva is not required by HUD.

Eva is a web-based tool built with R Shiny. This means:

  • Eva will only access your CoC’s data during your session, no CoC data is being retained or viewed by anyone besides you. Eva does retain metadata about the upload file itself, such as the name of your software vendor, your export dates, hash status, and data source information. This is collected for troubleshooting and tool planning purposes.
  • You can upload a zipped CSV Export of up to 200 MB. The file must be hashed.
  • You can stay up to date with the new features by visiting the Changelog tab.

Instructions

Eva works by uploading a hashed HMIS CSV Export.

Generate a hashed HMIS CSV Export from your local HMIS and store it in a secure location that you can easily find again. It must be a .zip file with 23 csv files in it.

  • A hashed export means that the personal identifiers are obscured when the export is generated.
  • The HMIS CSV Export has client-level data in it, so it must be stored in a secure location per HUD, state, and local rules and regulations.
  • If you are unsure how to generate your hashed HMIS CSV Export, please contact your vendor.

Once you have exported the correct file from your HMIS, you are ready to engage with Eva. Navigate to the 'Upload HMIS CSV Export' tab and follow the instructions there.

Want to explore Eva without uploading? Use Eva's Demo Mode by clicking the toggle at the top.

Demo Instructions

Need help?

Trouble-shooting tips:

  • Be sure you have generated the correct export. You cannot use an APR or LSA export file for use with this tool.
  • Eva is looking for a zip file and will extract the files for you, so unzipping your export is not necessary.
  • If your export is a .7z file, you must convert it to a .zip file. If you are not sure about how to do this, please contact your vendor.
  • If something is not working, please go to GitHub to check for known issues and/or enter any new issues or feature enhancement requests. To enter an Issue on GitHub, you must have an account. If you do not already have a GitHub account, you can sign up for one here.

Citations and Special Thanks

This project would not exist were it not for the existence of other quality, free and open source products. The following are citations for the products this app relies on.

The foundational code for the app was shared via AGPL license by the Coalition on Homelessness and Housing in Ohio (COHHIO).

R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria. R programming language.

Wickham et al., (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), 1686, Tidyverse package.

Chang W, Cheng J, Allaire J, Sievert C, Schloerke B, Xie Y, Allen J, McPherson J, Dipert A, Borges B (2021). _shiny: Web Application Framework for R_. R package version 1.7.1, R Shiny package.

Chang W, Borges Ribeiro B (2021). _shinydashboard: Create Dashboards with 'Shiny'_. R package version 0.7.2, shinydashboard package.

Special thanks to Square Peg Data, the CoCs who provided us with sample datasets to support programming.

Instructions

Upload hashed HMIS CSV Export

To upload your hashed HMIS CSV Export, click the 'Browse' button. Once you find the zip file on your computer, click on it to select it, and click 'Open' to begin the upload. Eva might take a few moments to process your selected file. Eva will first check to determine if the export is hashed. If it is not, Eva will reject the file with an error message, and clear Eva's memory. Eva will continue to do this until you upload a hashed HMIS CSV Export.

HMIS CSV Export File Structure Analysis

After confirming your export is hashed, Eva will review and process the file structure of your upload. The File Structure Analysis assesses the structural components of the uploaded .zip file and determines if it meets Eva’s file structure requirements, such as if the files have all the right names, columns, data types, and allowable values etc.

Once your upload is processed and Eva has finished assessing the file structure integrity of your upload, Eva will provide a pop-up message alerting you of your upload status. You can have either a successful upload or an unsuccessful upload based on the structural integrity of your HMIS CSV export. The key difference between a successful upload and an unsuccessful upload is if the upload has any High Priority File Structure Errors.

While any error identified during the File Structure Analysis represent components in the uploaded HMIS CSV export file that do not meet the most recent HMIS CSV Format Specifications, there are some file structural errors that are more relevant to the functionality of Eva.

  • High Priority File Structure Errors are file structure issues that will cause Eva to not work.
  • General File Structure Errors are file structure issues that will not impact Eva’s ability to work, but do not meet HMIS CSV format specifications.

If Eva identifies any High Priority File Structure Errors during the File Structure Analysis that prevent Eva from functioning, Eva will reject your upload and stop processing the export. You will thus not be able to assess the data quality of your upload or analyze the system performance of your homeless response system. For both successful and unsuccessful uploads, all identified file structure errors will display in the HMIS CSV File Structure Analysis panel, where you can download the details.

It is essential that you contact your HMIS vendor to resolve all High Priority File Structure Errors identified in the HMIS CSV Export File Structure Analysis, as well as any other structural issues which you feel need to be corrected. Not all structural issues found in this analysis will prevent the data from being accepted for analysis, so they may not require immediate attention. Once your vendor has addressed any High Priority File Structure Errors, you can attempt another upload.

Once you have uploaded a hashed and structurally sound .zip file, you will see a confirmation that your upload was successful, the date range of the files you uploaded, plus the date your Export was downloaded from your HMIS. You will then be able to assess the data quality of your upload and analyze the system performance of your homeless response system.

Loading...

HMIS CSV Export File Structure Analysis

Users should contact their vendor to resolve high priority errors identified in the HMIS CSV Export File Structure Analysis, as well as any other structural issues which you feel need to be corrected.

Instructions

To make Eva data quality analysis more useful at the local level, you can adjust the local settings to better analyze your data in a way that is meaningful to your CoC. To edit these, click to expand the relevant box below. If you do not edit them, the Assess Data Quality and View Client Counts pages will use the defaults listed. Please note, these local settings do not impact the System Performance Overview page.

These defaults do not imply any HUD recommendations. Please read the description in the Edit Local Settings tab for more information.

Outstanding Referrals

This check aims to help communities find Coordinated Entry (CE) Event referrals that may be missing a Result Date or may have been delayed in getting the client to housing. This check is only applied to CE Event referrals which are expected to have an associated Result and Result Date (4.20.2 responses 10-15, 17, 18. Please see the HMIS Data Standards for the complete list of CE Events.)

When a CE Event referral does not have a Result Date at the time the export is uploaded, Eva calculates how many days the referral has been open by looking at the number of days between the Referral Date and the date your upload was exported from your HMIS. Then Eva compares the length of each open referral with the 'Max Days' assumption entered in the input field below. If the referral is open longer than the expected timeframe, it is categorized as an 'Outstanding Referral.' This check is for all projects that have a relevant CE Event referral.

Data quality flags about Outstanding Referrals are categorized as Warnings, indicating that data should be reviewed for accuracy. It does not imply that any data should be changed.

In the field below, specify the maximum number of days a referral can stay open according to the CoC's Coordinated Entry Referral process. The value defaults to 14 days. (These defaults do not imply any HUD recommendations).

Long Stayers

This check aims to help communities find enrollments that may be missing an Exit Date. First, the tool calculates the number of days each enrollment has been open (meaning, the number of days between the Entry Date and the date your upload was exported from your HMIS.) Then the check uses one of two methodologies to identify Long Stayers.

For select project types, the check identifies the top % of longest stayers in each project type. For other project types, the check compares the length of each enrollment with assumptions entered about the expected maximum period of assistance envisioned for the project type. For the latter check, users can set the assumptions for each project type. All data quality flags about Long Stayers are categorized as Warnings and is a suggestion to verify that the identified clients are still active in these projects. It does not imply that any data should be changed.

Top 2% longest enrollments are flagged for the following project types:

  • Emergency Shelter - Entry/Exit
  • Safe Haven
  • Transitional Housing
  • Rapid Rehousing
  • Homeless Prevention

Top 1% longest enrollments are flagged for the following project types:

  • Permanent Supportive Housing
  • Permanent Housing with Services (no disability required for entry)
  • Permanent Housing - Housing Only

Enrollments active longer than the CoC-specified length of assistance targets are flagged for the following project types:

  • Emergency Shelter - Night-by-Night
  • Street Outreach
  • Other
  • Services Only
  • Day Shelter
  • Coordinated Entry

Below, you can specify the expected maximum period of assistance envisioned for the project type, meaning the timeframe after which you would want an organization to confirm the client is still active in the project. You can set these based on your current data or leave them at the defaults (these defaults do not imply any HUD recommendations).

Instructions

Client Counts Report

The Client Counts Report provides the number of households/clients who have been served in each project and their enrollment status at the time of CSV Export generation. This report can be used to verify that a project is up to date on their HMIS data entry by comparing counts reported with the number of households/clients that are known to be served in each project. Permanent housing projects can check that the number of households/clients who have not yet moved into housing is correct.

Inputs

Select a project from the drop list and adjust the the Date Range for the cohort of clients you want to see reported. The Date Range defaults to the date range covered by the HMIS CSV Export. Users are encouraged to edit the Date Range as desired to see metrics for timeframes within the Export period, such as the number of households/clients who exited during that timeframe with and without a Move-In Date. Note that setting the Start Date to the Export End Date will show the current status for all enrollments for the project.

Summary

The Summary panel provides a count of households/clients who have statuses of the following within the selected project:

  • Active No Move-In Date
  • Currently Moved In
  • Exited No Move-In
  • Exited With Move-In
  • Currently in Project
  • Exited Project

Client Counts Detail

In this panel you will see the Personal ID, Relationship to HoH, Entry Date, Move-In Date, Exit Date, and the Status for each client served by the selected project within the Date Range selected. The rows are ordered by Entry Date (oldest on top), Household ID (not visible), and Personal ID. This enables users to see the oldest enrollments first and groups clients in the same household together. All columns are searchable. For example, to find all enrollments with a Status of 'Active No Move-In Date', you can type 'act' in the Status search bar and the data table will react and filter in that way.

Download System-wide Client Counts

To download client count data for all projects in your HMIS CSV Export, click the System-wide download button. The download contains a Current tab limited to just the current date, a Date Range tab limited to the Date Range set, and a Detail tab with clients' PersonalIDs, Entry Date, Move-In Date (if applicable), Exit Date (if applicable), and project status.
to

Select Project

Client Counts Summary

Client Counts Detail

Instructions

Project Descriptor Data Element (PDDE) Check Summary

Once you have successfully uploaded an HMIS CSV Export, you will find a summary of each issue that was flagged in your data regarding your PDDEs. Please download the details by clicking the 'Download' button.

Guidance

For a description of each issue found, check the Guidance panel.

PDDE Check Summary


Loading...

Guidance

Instructions

Organization-wide HMIS Data Quality

Below, select the organization whose data quality you would like to check. The data shown will reflect the date range that you used to run your HMIS CSV Export. It will show data quality metrics from all Projects that are associated with that organization.

You can click the Download button to generate an Excel workbook with the selected organization's data quality errors. You can send these to authorized HMIS users at the selected organization so they can work on correcting their data. Feel free to modify, add, or remove anything as you see fit. For example, you may want your users to only address High Priority issues right now. You can easily remove any tabs that may distract your users from that goal. Please note that Overlaps will be shown in the 'Warnings' tab and again in the 'Overlap Detail' tab of the download. This is so your users have enough detail to track down each issue.

Note that protected personal information (PPI), such as Personal ID in combination with other data elements, is contained in the Excel downloads. Users must follow all applicable HMIS privacy and security policies when storing, transmitting, and disclosing files with client records.

Some definitions:

This app categorizes every issue it finds in your data set in terms of its severity.

  • High Priority Errors
    • Always indicates a data quality issue that can and should be fixed in HMIS.
    • Aim for 0 High Priority errors.
    • These errors affect multiple federal reports in a fundamental way and thus should be prioritized.
  • General Errors
    • Always indicates a data quality issue that can and should be fixed in HMIS.
    • Aim for 0 General Errors.
  • Warnings
    • May be a data quality issue, but may also be an unexpected situation that reflects reality.
    • Do not aim for 0 Warnings. It is ok and expected to have some warnings.
    • End users should check that any data being flagged as a Warning is accurate in their HMIS. If it is not accurate, then it should be corrected. If it is accurate, it should be left as is.

Regardless of an issue's categorization, users should never edit data that accurately reflects reality.

Organization-wide HMIS Data Quality Plots

For each type of issue (High Priority Errors, General Errors, and Warnings) you will find two plots: one graphing the number of issues by type, and one graphing the number of issues by project.

Top 10 Issues

Across all the projects within the selected Organization, this plot shows the top 10 issues identified. This can be useful in planning targeted HMIS training efforts.

Top 10 Projects

These plots show the top 10 projects within the selected organization with the highest number of issues identified. You can use this to help determing which projects may need extra assistance in addressing their data quality issues.

Download Organization-wide HMIS Data Quality Data

To download all of the client and enrollment related issues found in the selected Organization, click the Download button. This will give HMIS admins a way of communicating to an Organization what kinds of HMIS data quality issues they have.

Data Quality Summary

Data Quality Guidance

Instructions

System Performance Overview

The System Performance Overview page in Eva features three system performance charts: the System Flow Chart, the Client System Status Chart, and the System Demographics Chart. The charts display system performance data, pulled from your uploaded HMIS CSV export, from all HMIS Continuum projects, excluding homeless prevention projects. Eva uses the last 12 full months of data in the upload, which constitutes the report period. Note that some charts on this page may not display if the uploaded HMIS CSV export has less than 12 full months of data.

The purpose of the system performance charts is to use your HMIS data to (1) evaluate how effective your homeless system is in moving clients through the system and helping them reach permanent housing, and (2) help you understand the demographic composition of all clients served in your homeless system. Explore each of the charts using chart tabs beneath the Filters Menu. Under each chart tab is a Chart subtab for viewing the chart itself and an Information subtab. The Information subtab includes a "Chart Overview” section that provides guidance on how to read the chart, and some charts additionally have an "Interpretation Tips” section that can help you interpret their output.

Use the Filters Menu to explore system performance trends of clients in your homeless system with specific characteristics. This has two components:

  1. The universal filters, the top row of the Filters Menu, impact the data shown on all three visualizations on this page. Universal filters include Household Type, Level of Detail, Project Type Group, and Gender and Race/Ethnicity Methodology Type.
  2. The demographic filters, the bottom row of the Filters Menu, only impact the data shown in the System Flow and Client System Status charts. Demographic filters include Age, Veteran Status, Gender, and Race/Ethnicity.

Use the drop-down menus to select the characteristics of the system subpopulation you want to analyze. The default selection is all clients in your homeless system throughout the report period. To see system performance by households, select the "Head of Households only” level of detail. All filters (except one) are single-select, meaning you can only select one category at a time. For the Age filter, you can select multiple age ranges to explore.

The Gender and Race/Ethnicity Methodology Type selection only impacts the Gender and Race/Ethnicity filters. To learn more about methodology and demographic categories, please visit the Glossary accessible on Eva’s Navigation Menu.

Downloads

To support further systems analysis, local reporting, and presentations, Eva includes two download options. To generate an Excel workbook with the data for a specific chart, click the "Data Download” button while viewing the chart. To generate a PowerPoint slide deck with the chart image, click the "Image Download” button while viewing the chart.

Data Suppression and Data Security

To ensure the privacy and protection of individuals and small population groups, Eva uses varying levels of data suppression. If the total number of clients within a chart is less than 11, the chart will not display. When this happens, you may need to broaden your filter selections or upload a larger dataset to ensure there is enough data to view the chart. A chart that is not displayed cannot be exported in Excel or PowerPoint.

The data in the data download of a chart’s export will not be suppressed. Be careful how you save and share the tabular export. With smaller numbers, clients can become more identifiable in the data. Before you share the Excel export, feel free to modify, add, or remove anything as you see fit to preserve client anonymity.

Filters

Placeholder

Glossary

Instructions

This glossary provides definitions for the terms used throughout Eva's System Performance Overview page. You can review definitions of the terms by their focus, including:

  • System Performance Filters
  • System Flow Chart
  • Client System Status Chart

You can also search for a specific term using the search bar.

Changelog

This tab will list the most recent technical updates and changes to Eva. For more in-depth information on current and past issues, please go to GitHub.

Instructions

System-wide HMIS Data Quality

Use your System-wide Data Quality data to evaluate which organizations may benefit from additional assistance and where extra training may be needed. You can download this data to use for reporting to interested entities about your overall HMIS system data quality.

Click the Download button to generate an Excel workbook with the your entire system's Data Quality data. Feel free to modify, add, or remove anything as you see fit. For example, if you are sending this workbook to your CoC management, you may want to remove the tabs that have client-level data on them.

Review the plots below to identify the organizations that you want to examine more closely in the Data Quality > Organization-level tab.

Some definitions:

This app categorizes every issue it finds in your data set in terms of its severity.

  • High Priority Errors
    • Always indicates a data quality issue that can and should be fixed in HMIS.
    • Aim for 0 High Priority errors.
    • These errors affect multiple federal reports in a fundamental way and thus should be prioritized.
  • General Errors
    • Always indicates a data quality issue that can and should be fixed in HMIS.
    • Aim for 0 General Errors.
  • Warnings
    • May be a data quality issue, but may also be an unexpected situation that reflects reality.
    • Do not aim for 0 Warnings. It is ok and expected to have some warnings.
    • End users should check that any data being flagged as a Warning is accurate in their HMIS. If it is not accurate, then it should be corrected. If it is accurate, it should be left as is.

Regardless of an issue's categorization, users should never edit data that accurately reflects reality.

System-wide HMIS Data Quality Plots

For each type of issue (High Priority Errors, General Errors, and Warnings) you will find two plots: one plots the counts of issues and one plots the number of issues by organization.

Top 10 Issues

Across all of the organizations in your upload, this plot shows the top 10 issues identified in the data quality scan. This result can help to focus future end-user trainings and bring to light any potential considerations in your federal or local reporting and analysis.

Top 10 Organizations

These plots show the top 10 organizations across your system with the highest number of issues identified. You can use these plots to help determine which organizations may need extra assistance in getting their HMIS Errors/Warnings resolved.

Download System-wide HMIS Data Quality Data

To download all of the client and enrollment related issues found in your system, click the Download button. This will give HMIS admins a way of reporting to interested entities, such as your CoC leadership, a broader view of the state of your HMIS data quality.