Write For Me Good Dissertation Online
Niagara County
Write For Me Good Dissertation Online Henry Hudson Pkwy zip 10024
Calculate Your Price
Word count 275
Price for this order:
$12
Proceed to
Order

When can we help you?

Running late with the deadline for your work? Then we are your reliable assistant in paper help.

Get ready to ask for our assistance when you need essays, research or course works, reports, case studies, etc. Our experts have seen it all and are ready to start working on your assignment right away. Go for it!

Why trust us?

With over 6 years of experience in the custom writing service, our team of support agents, managers, editors and writers has got a lot of knowledge about everything that may be required by you. Here’s what you get for sure when cooperating with us:

  • Professional paper writer. Our native speakers are true masters at creating unique and custom works per the most detailed and even the vaguest instructions you have.
  • Quality writing. We check all works with several detectors. To make sure your work is 100% unique, add a Plagiarism Report to your order.
  • Convenience of your personal control panel. Signing in, you can get your expert in touch through a direct message, also place orders there, and see their statuses.

Everyone needs some paper help from time to time, because we are only human.

What’s included in the total cost?

Our prices start at $10 per page for works completed from scratch and from only $6 per page you need to be edited and proofread.

What factors influence the cost of our paper writing services? There are 5 of them:

  • Type of work
  • Deadline
  • Academic level
  • Subject
  • Number of pages

You’re a lucky client! Why? Because you never pay for everything. You have lots of freebies to go with every single assignment. They are:

  • title page
  • reference page (bibliography)
  • formatting
  • 3 free revisions
  • scan for originality
  • email delivery

Asking for our paper writing help, you don’t only pay us. We also pay you! You can receive up to 15% bonuses back and even earn money with our referral program.

Your full security

We understand that sometimes you may want your deeds to go unknown. That is why we guarantee your complete privacy and security with our paper help writing service. After registration, you receive a unique ID and that is the only thing along with your instructions visible to our experts. Only our support team will see all the details you provide to be able to contact you in case any questions arise and send you a happy birthday discount on your special day.

Our custom writing service is completely ethical and provides busy students with great resources for their assignments. In the modern world when we need to do a lot of things at the same time, it’s nice to know you can count on someone for back up. We are always here to create the needed sample or perfect your work through editing/proofreading or explain the solutions to any problems you may have. Find out how much more free time you can get with our writing help.

SUNY State-wide Colleges barkada kontra droga essay checker persuasive essay 4th grade examples of conductors Columbia College Rockland County, transformers prime cyberverse dreadwing review of literature cover letter for sound engineer W 94th Street zip 10025 solar system lesson plans for 4th graders Albany Pratt Institute sample academic cover letter english 36th Street, West zip 10018. Essex how to get your adhd kid to do homework W 153rd Street zip 10031 working outline for research paper apa examples Elmira College, Elmira mUSTERBeweRBUng bAnKkaUfFRAU AuSbILdUnG SparKasSE looking for someone to write my course work on literature now.

OPPORTUNITY PROJECT GOING INTO THE FUTURE. AND THIS IS A REALLY NEAT OPPORTUNITY BECAUSE, YOU KNOW, AS AMERICA'S DATA AGENCY, WE ARE COMMITTING TO PROVIDE THE TEAM AND THE INFRA INFRASTRUCTURE TO REALLY DRIVE THAT PROJECT FORWARD AND SHOW HOW WE CAN BE BOTH THOUGHT AND OPPORTUNITY LEADERS ON A VERY IMPORTANT SOCIAL JUSTICE PROGRAM PROGRAM. SO WE'RE PROUD OF IT. I THINK THE FACT THAT WE WERE ASKED TO LEAD ON IT IS PROBABLY A VERY GOOD SET OF VALIDATION THAT WE HAVE BEEN LEADING ON A LOT OF THE OPEN DATA ISSUES, THARNTION TO THE GUIDANCE THAT YOU -- THANKS TO THE GUIDANCE THAT YOU ALL HAVE BEEN GIVING US US. BUT ANOTHER THIRD REAL SET OF EFFORTS HAS BEEN OUR EFFORT TO LEVERAGE PRIVATE SECTOR INNOVATORS AND TO ADDRESS ISSUES DANA BOYD OBVIOUSLY HAS BEEN A LEADING THINKER ON FOR SOME TIME TIME. ONE OF THE ISSUES THAT WE HAVE BEEN REALLY CONCERNED ABOUT IS JUST MAKING SURE THAT OUR DATA GETS TO THOSE WHO AREN'T USING IT, TO THOSE WHO MAY NOT KNOW THAT THEY CAN USE IT TO SOLVE THEIR PROBLEMS, AND WE HAVE BEEN FOCUSED ON HIGHLIGHT HIGHLIGHTING DATASETS THAT MIGHT BE HELPFUL TO THEM, CURATEING THEM, GETTING THEM INSTRUCTIONS HOW TO USE IT, SPEAKING REGULARLY AND ELEVATEING THEM, BUT MOST IMPORTANTLY, WE CHALLENGE THE PRIVATE SECTOR AND CANNOT DO EVERYTHING, DWOANLT KNOW THE USER GROUPS -- WE DON'T HAVE THE ABILITY TO GO AND SCOPE EVERY SOLUTION, BUT YOU DO, SO HELP US. AND I REALLY COULDN'T BE MORE PROUD OF WHERE THAT HAS GONE. WE HAVE HAD A SERIES OF COMPANIES STAND UP AND SAY I WILL TAKE THIS PUBLIC RESOURCE, I WILL TAKE THIS DATA AND I WILL HELP TAKE IT THE LAST MILE OR I WILL CLEAN IT AND PREPARE IT AND GOOD JUST SO OTHERS WHO AREN'T USING IT CAN USE IT BETTER OR SO IT. AND JUST IN THE LAST COUPLE OF WEEKS, WE'VE HAD ANNOUNCEMENTS YESTERDAY ON A PROJECT DIRECTLY WITH KAGGLE WHERE THEY WILL BE PUTTING COMMERCE DATASETS BEFORE A DATA SCIENCE COMMUNITY OF 700,000 DATA SCIENTISTS. EPHESOFT IS A SMALL COMPANY THAT HAS OCR AND MACHINE LEARNING CAPABILITY AND THEY'RE TAKING DATA THAT IS IN TIPS, THEY'RE PROCESSING IT AND PUTTING UP THE UNSTRUCTURED AND STRUCTURED DATASETS FOR OPEN AND PUBLIC USE. SOCRATA, KEVIN, STOAPPED THE PLATE AND TOOK AN UNDERUTILIZED DATASET INVOLVING HEALTH HEALTH INSURANCE INFORMATION PUT IT UP ON THIS PLATFORM AND YOU'LL SEE LATER THE INSIETSZ AND USE YOU'RE ABLE TO GET BY BEING ABLE TO VISUALIZE AND CUT THE DATA VERY, VERY QUICKLY. THESE ARE THE KINDS OF THINGS THAT WE'RE TRYING TO TAKE THAT DATA THE LAST MILE AND MAKE USABLE PROJECTS AND PRODUCTS BY LEVERAGING EXPERTISE THAT WE DON'T HAVE. DATA DOT WOLVES IS DOING A SIMILAR THING, WE EVEN HAD WOLFRAM COMING TO THIS FLOOR AND POSTING AN OPEN CODING SESSION TO BE ABLE TO SOLVE PROBLEMS AND THERE ARE MORE. NEXT COUPLE MONTHS, YOU'LL SEAL ONE AFTER ANOTHER COMPANIES STEPPING FORWARD AS YOU GET THE FLYWHEEL TURNING AND THEY SEE THE OPPORTUNITY TO HELP TO REALLY LEVERAGE THEIR EXPERTISE TO GET THE MULTIPLIERL EFFECT. WE'RE REALLY PLOWED ABOUT WHERE THAT IS TAKING US. OUR EFFORTS ARE STARTING TO REALLY GET RECOGNIZED. FED SCU AWARDED OUR AWARD FOR FEDERAL LEADERSHIP, THE ACADEMY THAT WE RUN HAD FED PROGRAM OF THE YEAR, THE COMMERCE DATA USABILITY PROJECT WON INNOVATION OF THE YEAR. JEFF HIMSELF WAS NAMED AS ONE OF THINK THERE'S A FANCIER TITLE. ALAN, YOU CAN TELL US. BUT IT GIVES YOU A SENSE THAT OUR PROGRAM REALLY HAS BEEN LED BY VISIONARYIES LIKE IAN AND JEFF AND OTHERS WHO HAVE ELEVATED OUR ANOTHER LEVEL. WE HAVE ALSO REALLY RELIED ON OUR OUTSIDE ADVISORS, OUR OUTSIDE BOARD OF ADVISORS LIKE OUR CDAC, INSIDE THE COMMERCE DEPARTMENT WE HAVE BEGUN A DATA EXECUTIVE COUNCIL WHICH IS CAREER LEVEL LEADERSHIP ALL ACROSS THE DEPARTMENT THAT ARE INVESTED AND HAVE BEEN LEADING ON DATA, AND THEY'RE COMING TOGETHER REGULARLY WITH WORKING GROUPS AROUND CORE ISSUES TO KEEP THAT MOVEMENT GOING. AND WE HAVE BEEN VERY ACTIVE IN A PROJECT THAT IS LED BY THE WHITE HOUSE CALLED THE DATA CABINET, WHICH IS AN EFFORT BEST PRACTICES AND INSTITUTIONALIZE WHAT WE'VE BEEN MORE, BUT THERE ARE MANY, MANY LEADERS IN THIS ROOM WHO HAVE REALLY BEEN TRYING TO SHARE WHAT WE'VE DONE. WE'VE STARTED TO -- YOU KNOW, IT'S PAINFUL TO MOVE THE ROCK UP THE HILL, YOU HAVE A SHARE THAT KNOWLEDGE. WE WANT TO SHARE HOW YOU CAN DO IT FROM AN OPERATIONAL AND OTHER PERSPECTIVE. WE WANT TO SHARE HOW YOU CAN DO IT FROM AN ADVOCACY PERSPECTIVE. WE WANT TO SHARE WHAT WORKED AND WHAT DIDN'T, AND I THINK THERE'S A REAL OPPORTUNITY TO GAIN SCALE AND TO BRAND OURSELVES AS BEING AT THE CENTER OF THAT MOVEMENT BY DOING THAT. THE LAST THING I'LL COVER AT A VERY HIGH LEVEL IS OUR EFFORTS TO INSTITUTIONALIZE WHAT WE'RE DOING, TO REALLY FOCUS ON THAT BECAUSE IT'S BEEN IMPORTANT. WE HAVE BEEN REALLY FOCUSED ON THIS PERSPECTIVE. WHAT ELSE OUR LEADERSHIP GOING INTO THE NEXT YEAR? I FEEL GREAT ABOUT THAT, FIRST OF ALL. MY DEPUTY, BRAD BURKS OUR DEPUTY -- BRAD BURKS OUR DEPUTY UNDERSECRETARY IS SI TERRIFIC LEADER, COMPLETELY MY PARTNER ON BEING A STRATEGIC LEADER ON THE ISSUES OF DATA, AND HE IS GOING TO BE A TERRIFIC TRANSITION AND GETTING OUR NEW TEAMS UP TO SPEED. JEFF CHEN WHO HAD BEEN OUR CHIEF DATA SCIENTIST IS NOW OUR DEPUTY CHIEF DATA OFFICER AND WILL BE THE LEADER OF OUR COMMERCE DATA SERVICE AGAIN IN TRANSITION BECAUSE IAN AND I ARE BOTH POLITICAL APPOINTEES. WE HAVE FOCUSEDLY ON PROJECTS AND DOLLARS, MAKING SURE THAT WE HAVE THE FUNDING, AND WE HAVE REALLY FOCUSED ON OUR RISKS, OUR HIRING AND OTHERS. AH, SECRETARY. RIGHT ON CUE. SO I'M LOOKING FORWARD LATE ON TODAY TO SHARE THINGS WE'RE WORRIED ABOUT, OUR RISKS OF EX- EX-CURKS OUR RISKS -- RISKS OF EX-CURKS OUR RISKS ON HIRING, AND TO GET INPUT FROM YOU ON WHERE WE ARE ON INSTITUTIONALIZATION, RIGHT NOW I WOULD LIKE TO BRIEFLY INTRODUCE COMMERCE SECRETARY PENNY PRITZKER. PENNY FROM OUR PERSPECTIVE ESPECIALLY , YOUR FOCUS ON HAVING DATA BE PART OF YOUR CORE STRATEGIC PILLAR HAS BEEN A REALLY STRONG DRIVER OF WHERE WE ARE TODAY. SECRETARY PRITZKER HAD THE VISION TO MAKE THIS ONE OF OUR FOCUSES TO REALLY DRIVE US TO INNOVATE, TO PUSH, TO USE SOME OF THE TOOLS THAT THE PRESIDENT HAS GIVEN US IN HIRING TRARK THARTS -- IN HIRING AUTHORITIES ON ISSUES AROUND THE GOVERNMENT. ONE OF MY GAIFORT COMMENCEMENT SPEECHES WAS ONE IN SAID IN TOTAL FOR THE SPEECH, NEVER GIVE UP, NEVER GIVE UP, NEVER GIVE UP, NEVER GIVE UP, AND SAT DOWN. AND I WILL TELL YOU, OUR MANTRA SINCE I'VE TAKEN OVER THE JOB IS GET IT DONE AND INSTITUTIONALIZE, GET IT DONE, INSTITUTIONALIZE. AND IT'S REALLY DRIVEN US I THINK TO PLACES THAT WE CAN QUITE PROUD OF. WITH THAT, SECRETARY PRITZKER. >> Penny Pritzker: WELL, JUSTIN JUSTIN, THANK YOU VERY MUCH. GOOD MORNING, EVERYONE, AND THANK YOU VERY MUCH FOR FIRST OF ALL COMING TO THE FIFTH MEETING OF THE COMMERCE DATA ADVISORY COUNCIL, BUT MORE IMPORTANTLY, THANK YOU FOR ALL THE TIME THAT YOU'VE PUT IN TO HELP US. I'M GOING TO DO A LITTLE BIT OF A REVIEW OF WHAT WE'VE ACCOMPLISHED OVER THE LAST 18 MONTHS, BUT I WANT TO SAY TO YOU THAT YOUR GUIDANCE HAS BEEN INVALUABLE, AND IF YOU COME AWAY SAY, JUST KNOW HOW APPRECIATIVE WE ALL ARE OF THE WORK THAT YOU HAVE DONE TO HELP US. YOU KNOW, YOU'VE PROVEN AND THE CDAC HAS PROVEN OVER THE LAST 18 MONTHS TO BE AN INVALUABLE PARTNER TO ALL OF US. REALLY YOUR EXPERTISE ACROSS THE PUBLIC, THE PRIVATE AND THE NONPROFIT SECTORS, BRINGING ALL OF THAT PERSPECTIVE, AND THEN YOUR IDEAS HAVE HELPED TO MAKE US A STRONGER ORGANIZATION AND DO A BETTER JOB FROM BOTH DATA COLLECTION AND DATA DISSEMINATION. THAT IS A FUNDAMENTAL PART OF WHAT WE DO AT THE DEPARTMENT OF COMMERCE , BUT YOU'VE REALLY MADE A MEASURABLE -- YOUR INPUT HAS MADE A MEASURABLE DIFFERENCE IN HOW WE DO WHAT WE DO AND WHAT EXACTLY WE'RE DOING. SO THANK YOU VERY MUCH. I REALLY APPRECIATE IT. IF WE STEP BACK, ONE OF HIS ADMINISTRATION AND MAYBE ON DAY ONE, AS I UNDERSTAND, I WASN'T THERE, BUT LAUNCHED THE OPEN DATA GOVERNMENT INITIATIVE AND BASICALLY DIRECTING ALL THE FEDERAL AGENCIES TO MAKE DATA AS EASILY ACCESSIBLE AND AVAILABLE AS POSSIBLE. ONE THING THAT WAS EXTRAORDINARY THE AMOUNT OF DATA THAT SITS AT DIFFERENT PARTS OF THE DEPARTMENT OF COMMERCE, AND IT'S NOT JUST SITTING HERE, BUT EVERY DAY WE'RE CREATING NEW INFORMATION THAT HAS VALUE IF THERE'S A POSSIBILITY TO MAKE IT IT ACCESSIBLE, AND YESTERDAY I GOT A DEMONSTRATION OF THE SOCRA SOCRATA TOOL THAT WAS BUILT JUST IN A VERY SHORT PERIOD AND OVER THE LAST SEVERAL WEEKS HELPING US TO VISUALIZE, AND I KNOW -- THANK YOU VERY MUCH. I WAS GOING TO SAY THANK YOU BECAUSE I'LL PROBABLY NOT DO JUSTICE TO THE VALUE OF THIS TOOL, BUT I WAS VERY EXCITED ABOUT IT. IN FACT, I WAS FOOLING AROUND WITH THE WHOLE THING, BUT IT'S A TOOL EASILY UNDERSTANDABLE FASHION HEALTH INSURANCE ACROSS EACH COUNTY OF THE UNITED STATES, WHO HAS GOT IT BY LOCALE, BY RACE, BY GENDER, BY SO MANY DIFFERENT DIMENSIONS, AND THAT WAS PREVIOUSLY INACCESSIBLE DATA, A DATASET THAT CAME FROM CENSUS, AND IT'S SO EASY TO HAVE ACTION ACTIONABLE INSIGHTS FROM THAT INFORMATION THAT CAN BE USED BY POLICYMAKERS, BY ADVOCATES, BY GOVERNMENT MANAGEMENT, BY, YOU KNOW, INSURANCE COMPANIES, ET CETERA, ET CETERA. SO THANK YOU EXAMPLE OF THE POSSIBILITY OF WHAT HAS BEEN CREATED HERE, AND I REALLY HAVE TO THANK THE FOLKS ON OUR TEAM. YOU KNOW, IAN, WHEN WE DECIDED TO MAKE DATA ONE STRATEGIC PLAN FOR THE DEPARTMENT OF COMMERCE, WE BEGAN OFFICER AND THE CHIEF DATA SCIENTIST, AND SO IANR CALIN AND WE WOULDN'T BE HERE EITHER, AND AND FOR ME WHO IS NOT A DATA SCIENTIST OR DATA ANALYST OR AN EXPERT IN DATA, WITHOUT THIS KIND OF TEAM THAT CAME TOGETHER, YOU KNOW, BURTON AND OTHERS, THIS WOULDN'T BE POSSIBLE. AND FRANKLY ALSO WITHOUT THE COOPERATION OF THE CENSUS DEPARTMENT BECAUSE IT'S REALLY BEEN -- AND THE PATENT AND TRADEMARK OFFICE AND SO MANY OTHER PARTS OF THE DEPARTMENT HAS ALLOWED US TO DO A NUMBER OF WE LAUNCHED THE COMERGS DATA SERVICE -- THE COMMERCE DATA SERVICE, WHICH IS OUR OWN INTERNAL INCUBATOR THAT'S FOCUSED ON BUILDING WORLD CLASS SOFTWARE PRODUCTS AND TOOLS TO MAXIMIZE THE VALUE OF OUR DATA ACROSS OUR 12 BUREAUS, AND THEN THE SECOND THING WE'VE DONE IS CREATED THE COMMERCE DATA ACADEMY THAT'S TRAINING OUR OWN EMPLOYEES ON DATA SKILLS, WHICH I JUST LOVE THAT, WE'RE NOT GIVING PEOPLE FISH, WE'RE TEACHING THEM TO FISH. AND THIRD IS THE COMMERCE DATA USABILITY PROJECT ARE WHICH IS TO MAKE OUR DATA MORE ACCESSIBLE ACCESSIBLE, VALUABLE AND USABLE TO THE PUBLIC. THEN WE'VE CHALLENGED THE TALENT OUTSIDE OUR GOVERNMENT TO MINE OUR DATA, WHICH WE'RE DOING THROUGH MISH TIFS LIKE THE CITY SDK PROJECT, EMPOWERING JOURNALISTS, STUDENTS STUDENTS, PROGRAMS AND BUSINESSES TO EXPLORE WHAT COMMERCE DATA REVEALS ABOUT OUR OWN COMMUNITIES, AND FOR EXAMPLE EXAMPLE, AT OUR RECENT WASHINGTON HACK-A-THON, ONE PARTICIPANT USED THIS TOOL TO INVESTIGATE WHAT HAPPENS TO LOCAL SCHOOLS WHEN NEIGHBORHOODS INSIGHTS. ANOTHER EXPLOREED WHY PEOPLE TAKE ADVANTAGE OF EARLY VOTING SO THAT WE CAN MAKE OUR ELECTION PROCESS MORE EFFECTIVE. IN JUST EIGHT MONTHS, WE HAVE BUILT A ROBUST OPEN DATA INFRA INFRASTRUCTURE THROUGH PARTNERSHIPS, SOMETHING THAT YOU FOCUS OUR EFFORTS ON, AND PARTNERSHIPS NOT ONLY ACROSS GOVERNMENT BUT ALSO ACROSS THE PRIVATE SECTOR , AND MUCH OF THIS PROCESS IS BASED UPON YOUR RECOMMENDATIONS. YOU RECOMMENDED WE DRAMATICALLY INCREASE ACCESS TO OUR RAW DATA, AND I'LL GIVE YOU A COUPLE EXAMPLES OF HOW WILL WE'RE DOING ECONOMIC ANALYSIS LAUNCHED ITS OWN OPEN SOURCE LIBRARY FOR GDP AND OTHER KEY ECONOMIC FIGURES USING LANGUAGE R, SO THIS B EXE AR -- SO THIS BEAR LIBRARY, B-E-A-R, THE FEDERAL GOVERNMENT, WE WILL LOVE OUR ACRONYMS, THE BEARL LIBRARY ALLOWS ECONOMISTS AND DATA SCIENTISTS AND OTHERS TO COMBINE AND ANALYZE INFORMATION FROM MULTIPLE DATASETS MORE EFFICIENTLY AND THESE ARE JUST A MAKING OUR RAW DATA MORE AVAILABLE. THE SECOND THING RS RECOMMENDED IS THAT WE MAKE OUR COMMERCE DATA MORE USABLE , AND THE OPPORTUNITY PROJECT, BY LEEPGD THE OPPORTUNITY PROJECT, COMMERCE IS BRINGING FEDERAL AGENCIES, LOCAL GOVERNMENTS AND BUSINESSES LIKE ZIL LOWLO -- ZIL L ZILLOW AND RED FIN, OPEN DATASETS AND TOOLS THAT INCREASE ECONOMIC MOBILITY IN OUR COMMUNITIES. FOR EXAMPLE, IN CITIES LIKE DETROIT OR NEW ORLEANS OR PHILADELPHIA OR WAWRVE WASHINGTON, D.C., THE OPPORTUNITY PROJECT IS PROVIDING THE DIGITAL TOOLS THAT ENABLE AMERICANS TO FIND INFORMATION ABOUT SCHOOLS OR SOCIAL SERVICES OR JOB OPPORTUNITIES, REALLY CRITICAL INFORMATION TO HELP PEOPLE TAKE ADVANTAGE OF THE SERVICES THAT EXIST WITHIN THEIR RECOMMENDED THAT WE MAKE OUR COMMERCE DATA MORE ACCESSIBLE. SO THE U.S. PATENT AND TRADEMARK OFFICE HAS BEEN A LEADER IN THIS REGARD BY MAKING ITS 225 YEARS OF DATA EASIER TO ACCESS AND EXPLORE THROUGH ITS INNOVATIVE DEVELOPER HUB AND ITS LATEST PROJECT CALLED PATENT VIEW. YOU ALSO RECOMMENDED THAT WE MAKE DATA PART OF EVERY BUREAU'S INSTITUTIONAL MUSCLE MEMORY, AND WE'RE MAKING PROGRESS ON THIS. TODAY YOU'LL HEAR ABOUT THE SEARCH STRING ANALYSIS PRODUCT AND HOW IT ENABLES OUR PATENT EXAMINERS TO USE A MORE POWERFUL GOOGLE-LIKE SEARCH TOOL, AND THAT WILL HELP US ENSURE THAT THE PATENTS THAT WE AWARD GO TO INVENTERS AND INNOVATORS THAT TRULY ARE SUBMITTING GROUND- GROUND-BREAKING DESIGNS AND IDEAS, BY BETTER BEING ABLE TO SEARCH OUR OWN INFORMATION. YOU'RE ALSO GOING TO LEARN ABOUT HOW OUR ECONOMIC DEVELOPMENT ADMINISTRATION'S NEW GRANTS VIEWER IS HELPING OUR GRANTS MANAGERS IN GOVERNMENT BETTER VISUALIZE WHERE FEDERAL GRANT DOLLARS ARE GOING AND THE IMPACT COMMUNITIES. I COULD KEEP GOING FOR A VERY LONG TIME, BUT I WON'T DO THIS. BUT WHAT I WILL SAY IS YOU'VE PROVEN HOW VALUABLE YOU ARE AS A GROUP, AND HELPED US AND OUR TEAM AT COMMERCE DRAMATICALLY MODERNIZE AND EXPAND ACCESS TO OUR DATA. WHAT'S CRITICAL IS WE INSTITUTIONALIZE THE CDAC TO ENSURE THAT THIS INNOVATION DOESN'T STOP AND THAT IT CONTINUES WELL INTO FUTURE ADMINISTRATIONS. SO TODAY I KNOW THAT WE'RE GOING TO CONSIDER AND I WOULD ASK YOU TO THINK ABOUT HOW COULD CDAC BE STRENGTHENED GOING FORWARD? FOR EXAMPLE, WHAT SKILL SETSZ SHOULD WE BE LOOKING FOR IN FUTURE APPLICANTS TO THE COMMITTEE? YOU KNOW, HOW CAN WE MAKE THE BEST USE OF YOUR TALENTS AND EXPERIENCE? WHAT ARE PROCESS IMPROVEMENTS THAT WE MIGHT MAKE? MANY. AND HOW SHOULD WE ADDRESS THESE ISSUES? THERE'S LOTS OF WORK TO DO. I MEAN, I THINK WE SHOULD FEEL REALLY GOOD ABOUT THE PROGRESS, BUT WE KNOW WE'RE REALLY ONLY AT THIS SORT OF FIRST OR SECOND INNING OF -- I HAVE BASEBALL ON MY MIND, I'M A CUBS FAN, I MUST CONFESS, SO MY BASEBALL METAPHORS ARE SORT OF UP IN MY CONVERSATION RIGHT NOW, THERE'S JUST A LOT MORE THAT CAN BE DONE, AND EVERY DAY WE CONTINUE TO CREATE MORE DATA, 20 AND, YOU KNOW, WHAT WERN IS BIG DATA DOESN'T -- WHAT WE KNOW IS BIG DATA DOESN'T HAVE A LOT OF VALUE OR IT HAS LITTLE VALUE UNLESS WE MAKE IT ACCESSIBLE AND USABLE FOR OUR BUSINESSES, ENTREPRENEURS, FOR OUR GOVERNMENT LEADERS, AND FRANKLY FOR OUR CITIZENRY AT LARGE. BEFORE I CLOSE, I'LL ALSO WANT TO THANK ONE LAST PERSON, AUSTIN AUSTIN, YOU KNOW, AUSTIN, YOU'VE REALLY BEEN A KEEL DRIVER OF ALL OF THE EFFORTS AROUND BIG DATA IN YOUR ROLE AS CHIEF OF STAFF, AND I WANT TO THANK YOU FOR YOUR LEADERSHIP BECAUSE IT TAKES A VILLAGE FOR US TO DO WHAT WE'VE DONE, BUT YOU'VE HELPED COORDINATE AND ORGANIZE AND DRIVE US, AND THANK YOU VERY MUCH FOR THAT LEADERSHIP. SO AS MAKING OPEN DATA A CENTERPIECE OF OUR DYNAMIC ECONOMY, THERE'S LOTS OF WORK FOR US TO DO AND WE NEED YOUR HELP, BUT I KNOW THAT WITH A PARTNERSHIP WITH ALL OF YOU AND FUTURE CDAC'S, THAT WE'LL BE ABLE TO UNLOCK MORE OF OUR DATA FOR BENEFIT OF LEADERS IN ALL PARTS OF OUR COUNTRY, ALL BUSINESS OR IN GOVERNMENT OR THE NEWBORN PROFIT SECTOR, SO THANK YOU ALL VERY MUCH. >> SECRETARY, THANKS VERY MUCH. WITH THIS, I THOUGHT I WOULD TURN IT OVER TO DANIEL WHORKS IS OF COURSE, -- CO-CHAIRS OF CDAC AND WHO HAS BEEN A TERRIFIC PARTNER FOR ALL OF US, HONESTLY. I KNOW THERE WAS A LOT OF HOME HOMEWORK WE GOT FOR THE CDAC, I APPRECIATE EVERYBODY CONTRIBUTING SO MUCH AND GETTING THOUGHT YOU COULD LEAD US IN A CONVERSATION AROUND SOME OF THE RECOMMENDATIONS THAT YOUR COLLEAGUES HAVE MADE TO US. >> Daniel Castro: GREAT. THANK YOU, SECRETARY, FOR BEING HERE TODAY AND THANKS TO EVERYONE. I WHEN YOU SAY WORDS LIKE R, TALKING PROGRAM LANGUAGES, GETTING A VIDEO VIEW, R AND PYTHON, THAT WAS VIRAL. AND KIND OF GET TO THIS BIGGER PICTURE . COMPLM I'VE BEEN REFLECTING ON HOW WE GOT HERE, AND, YOU KNOW, I THINK THE FIRST STEP WAS, YOU KNOW, AS YOU MENTIONED, AUSTIN REALLY HELPING TOGETHER THE CDAC ITSELF, BUT IT WAS THE DATA PILLAR I THINK FIRST GOT EVERYONE HERE INTERESTED AND ASKING HOW DID THIS GET SO HIGH UP IN THIS GOVERNMENT AGENCY? WHAT ARE THEY DOING? THIS SOUNDS REALLY INTERESTING AND DIFFERENT THAN EVERYONE ELSE, AND WE WANT TO BE THAT FIRST MEETING AND THERE WERE A LOT OF BIG IDEAS. THE BIG IDEAS -- WHAT DO WE NEED TO DO, HOW CAN WE MAKE THAT BETTER? AND OVER THE LAST, YOU KNOW, FEW MONTHS, WE'VE HAD A NUMBER OF MEETINGS, WE'VE TRIED TO REFINE HOW WE DO THEM, WE'VE TRIED TO PROVIDE I THINK VERY DIRECT AND SPECIFIC FEEDBACK ON CERTAIN PROGRAMS THAT WE DON'T NEED TO GET INTO NOW. WE ALSO HAD SOME VERY CONSISTENT LARGE THINGS WITH ABOUT HOW DO WE MAKE THIS WORK AND HOW DO WE REALLY CHANGE THE FUTURE OF THIS AGENCY AND THE DEPARTMENT. SO THE FIRST RECOMMENDATION THAT WE HAD, WE SAT DOWN TO PUT TOGETHER A BRIEFING HERE, WAS INSTITUTIONALIZEING THE DATA INITIATIVE AND THIS IS THE THEME IT DOESN'T END HERE. SO THERE'S A LOT THAT WE CAN TALK ABOUT HERE. I THINK AS WE'RE TALKING ABOUT WHERE WE ARE RIGHT NOW IN TERMS OF THE TRANSITION, ONE OF THE BIG QUESTIONS IS HOW DO WE JUST MAKE SURE EVERYONE KNOWS ABOUT KIND OF THE GENIUS THAT'S HAPPENED HERE? HOW DO WE MAKE SURE THAT THIS IS WELL RECOGNIZED NOT ONLY WITHIN THE DEPARTMENT ITSELF BUT ACROSS FEDERAL AGENCIES, ACROSS, YOU KNOW, MEMBERS OF CONGRESS, THROUGHOUT THE ADMINISTRATION, AND EVEN MORE BROADLY IN THE PUBLIC. SO KIND OF ONE OF OUR KEY DISCUSSION POINTS AS WE MOVE FORWARD HERE. THE SECOND RECOMMENDATION WE HAVE IS AROUND ITSELF. I REMEMBER THE SECOND MEETING WHEN IAN CAME IN AND SAID I HAVE SOME BIG NEWS HERE AND STARTED WALKING US THROUGH WHAT WAS DONTA FIRST MEETING WITH THE COMMERCE DATA SERVICE, WE WERE ALL EXCITED ABOUT THAT INITIATIVE AND HAD A LOT OF FEEDBACK THEN AND IT'S BEEN GREAT TO SEE HOW IT'S EVOLVED WITH JEFF COMING ON AND THE OTHERS AS WELL AND THE WHOLE TEAM THAT WE MET AT THE LAST MEETING. THERE'S BEEN A NUMBER OF REALLY GOOD PROJECTS COMING OUT OF THAT, BUT CONSISTENTLY I THINK WE'VE ASKED TO SEE A FEW THINGS. ONE, WE SAID LET'S START SMALL, LET'S DO SOME PILOT PILOTS AND EXPERIMENT. WE ALSO SAID WE NEED TO HAVE OUT THERE REACH, HOW DO WE REALLY SHOW IMPACT. I THINK THAT'S SOMETHING WE WANT TO SPEND A LITTLE TIME TALKING ABOUT RIGHT NOW IS WHAT MIGHT THOSE MOONSHOT MOONSHOTS LOOK LIKE AND HOW CAN WE REALLY HELP RAISE THE VISIBILITY OF THIS PROJECT THROUGH THOSE TYPES OF INITIATIVES. AND THE DATA ACADEMY SPECIFICALLY IS SOMETHING THAT I THINK ALL OF US HAVE BEEN VERY EXCITED ABOUT, WORKFORCE ISSUES HAVE COME UP QUITE FREQUENTLY, AND WE'VE SEEN THE DATA ACADEMY AS SOMETHING THAT COULD BE MODEL THE THROUGHOUT THE FEDERAL GOVERNMENT AND WE WOULD LIKE TO RECOMMEND THAT THAT REALLY BE A KEY PART OF THE COMMERCE DATA SERVICE. BEYOND THAT, AGAIN GOING BACK TO THAT FIRST MEETING MEETING, THERE WAS THIS QUESTION ABOUT ARE WE USING DATA TO MEASURE WHAT WE'RE DOING WITHIN THE DATA SERVICE. SO THERE WERE QUESTIONS ABOUT WHAT KIND OF METRICS ARE WE USING, DO WE REALLY KNOW THE CUSTOMER, HOW DO WE ENGAGE WITH USERS, WHETHER THEY'RE BUSINESS USERS OR INDIVIDUAL CITIZENS, OTHER PARTNERS, AND WE'VE SEEN SOME EVOLUTION THERE AND I THINK THAT WILL CONTINUE TO BE AN AREA OF FOCUS IN OUR MEETINGS WHERE WE'RE ASKING QUESTIONS ABOUT HOW WE CAN DO THAT BETTER, HOW WE CAN BRING THE PRIVATE SECTOR TO BEAR HERE. IN YOUR REMARKS THIS MORNING YOU TALKED ABOUT SOME OF THE PARTNERSHIPS, THERE HAVE BEEN GREAT ONES WITH PRETTY MUCH THINK, AND THAT'S BEEN ONE OF THE BIGGEST SUCCESSES OF THE CDAC ITSELF AND THE RELATIONSHIPS IT CREATED, ALSO JUST THE FUTURE OF HOW THIS CAN WORK. THE COMMERCE DEPARTMENT HAS CLEARLY SAID WE'RE OPEN TO BUSINESS WHEN IT COMES TO DATA, COME PARTNER WITH US, AND JUST RECENTLY WITH THE NEW JOINT RESEARCH INITIATIVE, ALL THOSE REALLY INTERESTING, INNOVATIVE APPROACHES TO PARTNERING WITH THE PRIVATE SECTOR I THINK HAVE OPENED NEW OPPORTUNITIES. WE'VE AS WELL. FINALLY, THERE'S THE CDAC ITSELF AND THE ROLE OF THE CDAC GOING FORWARD AND HOW WE CAN CONTINUE TO BE USEFUL. I THINK IN THIS AREA WE CERTAINLY HAVE SOME SUGGESTIONS, WE TRIED TO, I THINK EVERYONE IN THIS ROOM HAS SEEN HOW WE'VE EVOLVED THE AJEAN DA, ITERATED EACH TIME TIME, BEING ABLE TO DIG IN MUCH DEEPER ON CERTAIN PROJECTS NOW THAT WE HAVE MORE KNOWLEDGE, BUT TOOL TO BRING NEW OUTSIDE INSIGHTS HERE. SO LET ME STOP THERE BECAUSE THIS IS A VERY TALKATIVE GROUP AND WE WANT TO MAKE SURE EVERYONE CAN WEIGH IN. BUT THOSE ARE OUR MAIN RECOMMENDATIONS HERE. I THINK VARIOUS PEOPLE MIGHT WANT TO WEIGH IN ON SPECIFICS, SO IF YOU TENT UP, WE'LL JUMP INTO THAT DISCUSSION. EITHER ONE. GO AHEAD. >> I'M SO GLAD COMMERCE IS TAKING DATA SO SERIOUSLY, AND I THINK IT'S FANTASTIC, AND ONE OF THE THINGS THAT I'M WATCHING THE INDUSTRY STRUGGLE WITH, WE'VE BEEN STRUGGLING WITH SECURITY ISSUES FOR A VERY, VERY LONG TIME, AND I KNOW THAT THEY'RE PART OF YOUR CONVERSATION AND TRYING TO THINK THROUGH BECAUSE EVEN AS GOOD WORK IS BEING DONE, THERE'S A LOT MORE TO BE DONE FOR EACH AND EVERY ONE OF US. BUT THERE'S ANOTHER THREAT THAT I THINK IS COMING THAT I DON'T THINK WE'RE PAYING ENOUGH ATTENTION TO AND I THINK THAT THE INDUSTRY IS STRUGGLING WITH IT, AND I WANT TO RAISE IT AS SOMETHING I THINK YOU NEED TO START IT STRUGGLING WITH AS WELL WELL, WHICH IS THE DEGREE TO WHICH DATA IS STARTING TO GET MANIPULATED. ONCE DATA IS TAKEN SERIOUSLY, ONCE IT HAS THE LEVEL OF POWER AND THE LEVEL OF OPPORTUNITY AND BUSINESSES DE DEPEND ON IT, WE SEE A MOMENT WHERE PEOPLE ARE STARTING TO MANIPULATE IT FOR A VARIETY OF DIFFERENT REASONS, ECONOMIC, POLITICAL, SOCIAL, AND ONE OF THE CHALLENGES IS THAT I DON'T THINK THAT WE HAVE THE STRUCTURES IN PLACE TO EVEN KNOW MANIPULATED, AND I DON'T KNOW THAT WE HAVE THE MINDSET TO THINK THROUGH WHY SOMEBODY WOULD TRY TO MANIPULATE DATA. WE CAN LOOK AT THE POLITICAL REASONS FOR MANIPULATEING CENSUS. SOME OF THESE THINGS WILL BE INVISIBLE. I'M CURIOUS FROM THE OTHER PERSPECTIVE, WHAT DO YOU THINK COMMERCE IS AND CAN DO TO START PREPARING NOT JUST FOR THE SECURITY RISKS, WHICH IN SOME WAYS ARE OBVIOUS AND CHALLENGING CHALLENGING, BUT FOR THE MANIPULATION RISKS? PR. >> GIVEN THAT WE RUN A LOT OF STATISTICAL AGENCIES, WHY DON'T I TURN THAT QUESTION OVER. >> I THINK IT'S A QUESTION WE FOCUS ON ALL ACROSS OUR AGENCY. AGENCIES AND SCIENTIFIC AGENCIES WITH REAL QUESTIONS THAT WE HAVE TO BE ABLE TO ANSWER AT ANY TIME AROUND SCIENTIFIC INTEGRITY, STATISTICAL INTEGRITY, THERE ARE OUTSIDE RESEARCHERS THAT ARE REALLY LOOKING AT OUR DATA THAT GIVE US A LOT OF CONFIDENCE. THAT DOESN'T MEAN WE'RE SILENT ABOUT IT, AND WE OBVIOUSLY UNDERSTAND THERE'S RISKS AND PEOPLE WHO MIGHT WANT TO MANIPULATE OR GET INTO THE SYSTEM, BUT LET ME GIVE YOU AN EXAMPLE. ON THE STATISTICAL FRONT ALONE, WE HAVE CAREER LONG LONG-TERM STATISTICIANS THAT WORK ON OUR EFFORTS TO GENERATE OUR CORE ECONOMIC STATISTICS. IT'S INSULATED IN MANY, MANY DIFFERENT WAYS FROM ANY POLITICAL INTERFERENCE, EVEN WHEN WE CALCULATE GDP, FOR EXAMPLE, THE CALCULATIONS ARE COMPARTMENTALIZED SO THAT EACH GROUPS IN BEA HAS THEIR VIEW OF THEIR LINE, BUT IT COMES TOGETHER ONLY AT THE VERY END IN A WAY THAT THE CAREER TEAM LEADS THAT AND HAS LED IT FOR A LONG TIME AND THEY GET VISIBILITY INTO WHAT THE NUMBER IS ONLY AT THE LAST MINUTE AND POLITICAL LEADERSHIP, FOR EXAMPLE, ON THAT NUMBER ONLY GETS IT RIGHT BEFORE IT'S PUBLICLY RELEASED, BUT WE ACTUALLY GET IT AFTER IT'S GONE TO JOURNALISTS. THEY DO A LOT TO MAKE SURE THAT THE STATISTICAL PROCESSES THAT WE USE ARE PUBLICLY KNOWN, WELL AVAILABLE, CAN'T BE CHANGED UNLESS YOU GIVE NOTICE THREE TIMES BEFORE THEY'RE RELEASED SO THAT YOU GET COMMENT AND INSIGHT INSIGHT, AND WE HAVE ALL, INCLUDING THE SECRETARY HAS REALLY MADE THIS CLEAR, IF YOU SEE A PROBLEM, IF YOU SEE SOMETHING WHERE SOMEBODY IS TRYING TO GAIN THE SYSTEM, LET US KNOW. WE HAVE AN INSPECTOR GENERAL WHO IS VERY AWARE OF THINGS LIKE THIS, AND THERE ARE LOTS OF DIFFERENT OUTLETS WHERE YOU CAN RAISE YOUR HAND. I CAN TELL YOU NOAA AND HAVING WORKED WITH THEM, ED CONCERNS KEARNS IS HERE, HAVING WORKED WITH BIG DATA, TAKES THE ISSUE OF SCIENTIFIC DATA VERY SERIOUSLY INCLUDING HAVING OUR LEADER SHICH AT NOAA STAND UP WHEN THERE WAS SOME SUGGESTION OF GETTING INTO INFORMATION THAT MIGHT CHILL HONEST AND OPEN INFORMATION AND SHARING OF INFORMATION. YOU'VE SEEN LITERATURE ACROSS THE DEPARTMENT BEING WILLING TO STAND UP AND SAY NO, WE'RE GOING TO TAKE THOSE BULLETS, YOU NEED TO BE ABLE TO AS SCIENTISTS SHARE OPENLY WHAT YOU NEED TO DO TO GET JUROR JOB DONE, AND IF -- TO GET YOUR JOB DONE, AND IF THERE'S SOMEBODY WHO IS CHALLENGING OUR PROCESS, OUR LEADERSHIP AT NOAA, OUR LEADERSHIP AT THE DEPARTMENT HAS BEEN WILLING STOONLD UP. ON A LOT OF OUR PROJECTS, WE ARE CHALLENGEING THE PRIVATE SECTOR TO LET US KNOW IF WE GOT IT WRONG. ONE OF OUR COURSE STATISTICAL PROJECTS, AS YOU ALL CHALLENGES WHEN WE'RE LOOKING TO THE GOAL OF TRYING TO USE PRIVATE DATA SOURCES, WE CAN TALK ABOUT THAT LATER, AND OUR OWN STATISTICAL PROJECTS, WE OPEN UP THE UNDERLYING ANALYSIS AND WE INVITE RESEARCHERS, COME AND TEST IT, MAKE SURE WE GOT IT RIGHT. YOU'LL SEE THE SAME ON THE SCIENTIFIC FRONT, WHEN YOU SEE -- YOU'LL SEE OUTSIDE EXPERTS THAT OUR TEAM ENGAGES WITH IN AN OPEN WAY, IN A SCIENTIFICALLY INTEGRATEED SOUND OUR ANALYSIS, QUESTION IT, USE THE SCIENTIFIC METHOD. SO WE'RE CAREER-DRIVEN LINE WITH A LOT OF PROTECTIONS. I UNDERSTAND IN THE MACRO THERE ARE STILL THINGS WE'RE HOPING THAT THE OUTSIDE EXPERTS WHO COME IN AND TEST OUR >> WHAT YOU'VE LEARNED OVER MANY YEARS OF DOING THIS, THE INDUSTRY IS STRUGGLING WITH IT, AND THEY CAN ACTUALLY LEARN A LOT FROM WHAT YOU HAVE DONE, WHAT HAS WORKED AND HA HAS WHAT HAS NOT NOT. >> THAT'S GREAT. >> THAT'S A BIG IDEA, A GREAT IDEA, FOR US TO TAKE THE LESSONS >> AND WHAT PROCESSES TO ENSURE THAT. WE'RE ALWAYS OPEN TO IMPROVING OURSELVES AS WELL, BUT WE'VE LEARNED A LOT BECAUSE THERE'S SO MUCH THAT RELIES UPON COLIN: I THINK THE COMMERCE DEPARTMENT MAY BE AT THE CORE OF SOLVEING THE PROBLEM . AS WE OPEN MANY OF THESE NEWLY INTERFACES, IF -- NEW INTERFACES INTERFACES, IF WE CAN APPLY SOME OF THE DEEP LEARNING TECHNIQUES THAT EXIST OUT THERE AND TIE SOME OF THOSE TO A VARIETY OF SOURCES OF DATA, WE CAN ACTUALLY CORRELATE AGAINST ANYTHING SOMEONE SAYS. YOU KNOW, IT COULD BE CENSUS DATA, PATTERN DATA, AND THE CORRELATION OF THOSE COULD ACTUALLY SHOW THAT THIS DATA WAS MANIPULATED, THIS DATA IS DIVERGENT. SO I THINK THERE MAY BE A NUMBER OF WAYS THAT WE COULD BE AT THE HEART OF LOOKING AT THESE THINGS BALLS I HAVE NO DOUBT THAT IT WILL OCCUR TO BEGIN TO DO THAT USING DEEP LEARNING OR REASON FORMATIONED LEARNING, I THINK WE COULD PROVIDE ANOTHER SORT OF VAST STRENGTH FROM THE WORLD DELIVERED FROM THE GOVERNMENT HERE AND THE COMMERCE DEPARTMENT DEPARTMENT. SOMETHING TO THINK ABOUT HERE. >> Penny Pritzker: ONE OF THE BIG BENEFITS WE HAVE IS AS WE'RE TRYING TO MODERNIZE AS WELL AS ACQUIRE NEW TECHNOLOGY, TECHNOLOGICAL CAPABILITIES, IS WITH NTIS WE HAVE THE OPPORTUNITY TO PARTNER WITH DIFFERENT ORGANIZATIONS, SO AS YOU'RE THINKING ABOUT RECOMMENDATIONS FOR US, ALSO BRINGING INTO YOUR CONSCIOUSNESS THAT AS THE CAPABILITY THAT WE'VE GOT. CR. >> WE ARE LOOKING, AS THE SECRETARY SAID, ALL OPTIONS ON MODERNIZEING OUR DATA STACK AND TECHNOLOGIES. I CAN'T TELL YOU HOW IMPORTANT THINGS LIKE AI WILL BE AND ALL OF THESE TECHNOLOGIES, INCLUDING AI, OPENING OUR DATA, WE'RE ALSO VERY CONSCIOUS OF MAKING SURE THE PROTECTIONS ARE IN PLACE TO MAKE SURE IT'S DONE RIGHT. SO DEIDENTIFICATION OF THE CENSUS, JOHN AND HIS TEAM HAVE ONE OF THE LEADER EXPERTS ON DE DEIDENTIFICATION, FOR EXAMPLE, AND JOHN ABOUD HAS BEEN REALLY IMPORTANT. >> I ALSO THINK RAPID DATA IS IMPORTANT, IF I'M GOING TO TRY TO CHANGE DATA ON CROPS, IF I KNOW ABOUT WEBER, I CAN TELL IF THE DATA HAS BEEN MANIPULATED, SO THE BREADTH OF THE DATA GIVES IT A CERTAIN POWER. >> SECRETARY, I JUSTMENT TO THANK YOU FOR BEING HERE -- I JUST WANT TO THANK YOU FOR BEING VOLUMES FOR THE LEADERSHIP THAT THIS DEPARTMENT HAS IN LOOKING FORWARD TO OUR DATA FUTURE, AND I HATE TO SORT OF TAKE AWAY ANY ATTENTION, BUT I'M CURIOUS ABOUT LIKE WHAT YOU'RE ABLE TO SEE FROM YOUR PERSPECTIVE IN TERMS OF OTHER AGENCIES ADOPTING A SIMILAR SORT OF FRAMEWORK AND GETTING AS ENGAGED AS COMMERCE HAS BEEN, BECAUSE THIS HAS BEEN A REALLY EXCITING GROUP I THINK FOR ALL OF US TO BE PARTICIPATING IN, AND REALLY I WOULD LIKE TO SEE IT BLOSSOM ACROSS OTHER AGENCIES AS WELL. DO YOU HAVE A SENSE OF WHAT OTHER AGENCIES ARE DOING? >> Penny Pritzker: THERE'S A DATA CABINET THAT EXISTS CREATED BY THE WHITE HOUSE, AND WHAT I'M PROUD OF IS I THINK THE DEPARTMENT OF COMMERCE HAS GONE FROM LET'S CALL IT MAYBE A LAGGA LAGGARD MEMBER TO A LEADING MEMBER, WHICH I THINK IS REALLY IMPORTANT. OBVIOUSLY THERE'S OPPORTUNITIES ON THAT CABINET, E EPA, DOL, DEPARTMENT OF LABOR, AND OTHERS, AND WHAT WE'RE TRYING TO DO IS HAVE BETTER COLLABORATION, BUT THIS IS WHERE YOU CAN MAKE RECOMMENDATIONS AS TO GOING FORWARD, YOU KNOW, HOW DO WE PLAY A GREATER INTERAGENCY LEADERSHIP ROLE IN THE DATA CABINET. SO THOSE ARE THINGS FOR MAYBE YOU TO HELP US ON WHAT WE OUGHT TO FOCUS ON IN THAT RESPECT. FIRST WE HAVE TO GET OUR HOUSE IN ORDER AND I THINK WE'VE DONE A LOT OF THAT, BUT, YOU KNOW -- THANK YOU. >> KATHY. >> WE ALREADY TALKED A LITTLE BIT ABOUT MOONSHOT AND I KNOW US USPTO WAS INVOLVED IN MAKING A DIFFERENCE FOR THE CANCER MOONSHOT AND I WONDER WHAT OTHER MOONSHOTS MIGHT BE RELEVANT FOR BUNDLING SOME OF THE GROUP FORCES HERE. ONE CO-TOMS TO MIND IS JOBS, I THINK IT'S VERY, DATA SCIENCE AND ALSO HIGH-TECH JOBS AND I THINK IT'S NOT JUST ACADEMIC SETTINGS THAT ARE ALWAYS INTERESTED TO HIRE PROGRAMMERS AND NOT ALWAYS SUCCEEDING IF THEY'RE COMPETEING MORE WELL TRAINED WORKFORCE AND OF COURSE ALSO GOVERNMENT WHICH HAS TO FILL THESE IMPORTANT ROLES THAT YOU NEED FOR THE CDAC THE CDAC EFFORT, FOR INSTANCE. YESTERDAY I ATTENDED A WORKSHOP ON GRADUATE STUDENT EDUCATION INVESTMENTS IN STEM CAREER EXPERIENCE AND OUTCOMES AND THERE'S A MAJOR INTEREST TO SHOWCASE A DATA DRIVEN WAY WHAT THESE INVESTMENTS RESULT IN IN TERMS OF PERFORMANCE WITHIN THE UNIVERSITY SETTING BUT ALSO IN LIFE, LIFE BALANCE ALSO, NOT JUST PROFESSIONAL SUCCESS. MUCH OF THE DATA THAT'S NEEDED TO REALLY MAKE A DIFFERENCE IS IN THE CENSUS BUREAU, IT'S IN THE DEPARTMENT OF LABOR, AND IT'S ALSO AMONG THE COMPANIES WHICH ARE HERE AND OTHERS WHICH ARE NOT HERE AROUND THE TABLE. SO I THINK THAT COULD BE A VERY, VERY VALUABLE EFFORT ESPECIALLY IF YOU CAN COMMUNICATE THE RESULTS OF THESE ANALYSIS TO EVERYONE OUTSIDE OF THIS BUILDING SO THAT THEIR TRAJECTORY MORE INTLGT INTELLIGENTLY, USING BRICK AND MORTAR UNIVERSITIES LIKE INDIANA UNIVERSITY WHERE I WORK AND OTHERS BUTTER ALSO USING THE ONLINE RESOURCES AND USING OF COURSE THE COMMERCE DATA ACADEMY A VERY, VERY VALUABLE MOONSHOT TO DISCUSS. ARE. >> I WOULD JUST SAY A COUPLE THINGS. PLACES WHERE WE USED OUR DATA, FOR EXAMPLE, WE HAD A HACK-A-THON CALLED HACK THE PAY GAP THAT CREATED TOOLS TO HELP BOTH INDIVIDUALS AND ORGANIZATIONS UNDERSTAND THE DISPARITYIES WITHIN THEIR OWN ORGANIZATIONS PAY AS WELL AS HELPING TOOLS THAT GOT CREATED THAT HELP ONE ADVOCATE FOR MORE EQUAL PAY, A NUMBER OF TOOLS CREATED THAT WERE REALLY VALUABLE. IN TERMS OF MOONSHOT IDEAS, WE'RE ALWAYS LOOKING FOR NEW PROJECTS AND NEW THOUGHTS AS GREAT INSIGHTS. THE OPPORTUNITY PROJECT IS OBVIOUSLY SERVING IN SOME RESPECTS TO HELP US THERE, BUT WE NEED HELP DOING TWO THINGS REALLY. ONE, AS IT RELATES TO WORKFORCE TRAINING AND EDUCATION, CLARIFYING WHAT IS THAT BOTH CONTENTWISE AND THEN HOW DO WE GET THE WORD OUT, AND THEN IN GENERAL, WE NEED HELP AMPLIFYING WHAT WE'RE DOING DOING. >> THE WHROO COMMUNITY, SOMETIMES WE HEAR VOICES IN CONGRESS AND ELSEWHERE SAYING WHY ISN'T N ON AA THE WEATHER SERVICE CHARGING COMPANIES FOR THE DATA TO PRODUCE? SO MY QUESTION IS, DO YOU ANTICIPATE THE PRESSURE TO MONETIZE COMMERCE DATA, DO YOU ANTICIPATE YOUR RESPONSE TO THAT? >> Penny Pritzker: YOU HAVE TO ANTICIPATE THE PRESSURE AS FEDERAL DOLLARS ARE SCARCE AND THE QUESTION I WOULD PUT BACK TO WE'RE TRYING TO DO IS TAKE A PUBLIC GOOD AND MAKE IT MORE AVAILABLE TODAY. THE QUESTION BECOMES ARE THERE SERVICE MODELS MAKE SENSE, AND HOW DO THEY MAKE SENSE IN THE CONTEXT OF THE FACT THAT THERE ARE SOME PUBLIC GOODS THAT WE NEED TO MAKE SURE ARE AVAILABLE AND AFFORDABLE LIKE THE WEATHER, WHICH IS ABSOLUTELY CRITICAL NOT JUST SO THAT WE KNOW WHETHER WE NEED A JACKET OR LIVES AND PROPERTY BECAUSE WE HAVE MANY SEVERE WEATHER EVENTS THAT CAUSE BILLIONS AND BILLIONS OF DOLLARS OF DAMAGE AS WELL AS, UNFORTUNATELY, AT TIMES TAKE LIVES, SO WE NEED TO HAVE THAT BALANCE OF OUR ROLES, IF YOU WILL. BUT I THINK THAT'S SOMETHING THAT MERITS MORE CONVERSATION AMONG THIS GROUP BECAUSE I THINK THERE WILL BE A LIMIT, THERE'S A LIMIT TO OUR RESOURCES AS TO WHAT WE CAN DO. WHAT WE'RE TRYING ON DEMONSTRATE TOFER ONE IS WHAT'S POSSIBLE, BUT WE'LL ONLY BE ABLE TO BE ON THIS WITH THE RESOURCES THAT WE HAVE. I DON'T KNOW IF YOU THINK >> NO, NO, HONESTLY, WE THINK ABOUT THE ISSUES AROUND AND PRESSURE WE MAY GET TO CHARGE AND I THINK WE'RE TRYING PROVE OUT THAT THE MULTIPLIER EFFECT FROM GETTING THE DATA OUT WILL GROW THE ECONOMY, WILL MULTIPLY AND SAVE LIVES IN A WAY THAT PROVES IT UP. BUT I THINK IT WILL BE AN ISSUE. >> Penny Pritzker: I'M GOING TO HAVE TO GO IN A FEW MINUTES, SO I WANT TO MAKE SURE, I JUST WANTED TO RESPOND, IF YOU DON'T MIND, TO YOUR RECOMMENDATIONS, IF THAT'S OKAY. YOU KNOW, FIRST OF ALL, IF I CAN JUST TAKE A MINUTE TO DO THAT, INSTITUTIONALIZEING OUR DATA IS ABSOLUTELY A PRIORITY, AND WE'RE COMMERCE DATA SERVICE HAS SUFFICIENT FUNDING INFRA INFRASTRUCTURE AND CONCRETE BUREAU-DRIVEN PROJECTS SO THAT WE GO FORWARD. WE ALSO NEED TO FOCUS ON NOT ONLY SUPPORTING OUR SURE THAT WE GET OUR PRODUCT OUT OUT. A PLACE WHERE WE'VE MADE PROGRESS BUT THERE'S MUCH MORE WORK TO BE DONE I THINK IS MARKETING AND BRANDING OUR WORK. SO IT'S CRITICAL FOR OUR LONG-TERM SUCCESS THAT THIS EFFORT IS AMPLIFIED, SO THOUGHTS AND IDEAS ABOUT HOW TO DO THAT WOULD BE USEFUL. WE'VE CREATED THREE ORGANIZATIONS INTERNALLY THAT WILL DESCREANSD ADMINISTRATIONS -- TRANSCEND ADMINISTRATIONS, FIRST THE EXECUTIVE COUNCIL, LEADERSHIP ACROSS THE DEPARTMENT, THE ECONOMICS STATISTICS IMPROVEMENT BOARD, WHICH IS WHEREBY HE A AND CENSUS DIRECTORS MEET TO DISCUSS WAYS TO BETTER COLLABORATE AND IMPROVE OUR STATISTICAL PROJECTS PROJECTS, AND THIRD IS THE COMMERCE DATA SERVICE WHERE IT'S NOW PERMANENT AND SUPPORTED CAN DO, BUT THESE ARE MAJOR STEPS WE'VE TAKEN TOWARDS INSTITUTIONALIZATION. AS IT REELTSES TO -- AS IT RELATES TO THE COMMERCE DATA SERVICE, WE'RE VERY PROUD OF THE SERVICE. IT'S NIMBLE, IT'S A START-UP. THAT HASN'T BEEN EASY TO GET UP AND GOING, AND WE'RE PROUD OF, FRANKLY, THE TEAM THERE AND THE VOLUME OF WORK THAT THEY'VE PRODUCED WITH FRANKLY A LIMITED NUMBER OF FOLKS, IT'S REALLY EXTRAORDINARY. THAT'S PERMANENT PERMANENT. WE NOW ARE FUNDING FROM THE BUREAUS AND OFFICES ACROSS THE DEPARTMENT THAT RECOGNIZE THE VALUE OF THE COMMERCE DATA SERVICE AND SO THEY'RE WILLING TO SUPPORT IT. WE HAVE A POLITICAL CHIEF DATA OFFICER AND A CAREER DEPUTY CHIEF DATA OFFICER AND THEY HAVE A GREAT TEAM, SO THAT HELPS, AND THEIR TEAM OF DATA SCIENTISTS, ENGINEERS AND CODEERS IS CRITICAL CRITICAL. BUT TO BE HONEST, AND HIRING IS A REAL CHALLENGE. WE HAVE OPEN POSITIONS FOR A HEAD OF ENGINEERING DATA SCIENCE, SENIOR ENGINEER, A SENIOR DATA SCIENCE LEAD, WE'RE LOOKING FOR FOLKS WHO WILL COMMIT LIKE ONE TO THREE YEARS APPOINTMENTS, AND WE WOULD LIKE TO SEE A STEADY STREAM OF FOLKS COMING. YOU COULD HELP US GET THE WORD OUT. FRANKLY, YOU SEE THE EXCITING PRODUCT THAT COMES OUT OF THIS SMALL BUT MIGHTY ORGANIZATION AND THE EXCITING WORK THAT THEY CAN DO AND THE PARTNERSHIPS THAT EXIST WITHIN THE DEPARTMENT AS WELL AS EXTERNALLY, SO I HOPE YOU'LL HELP US GET THAT WORD OUT ABOUT THE OPPORTUNITY. YOU KNOW KNOW, MY DREAM IS ULTIMATELY THERE'S A FLOW OF PEOPLE BOTH IN THE DIGITAL AND IN THE DATA WORLD THAT ARE COMING FROM THE PRIVATE SECTOR OR NONPROFITS INTO GOVERNMENT FOR A FEW YEARS AND GOING OUT AND COMPANIES AND OTHER ORGANIZATIONS VIEW THAT AS INTERFACE WITH THEIR TALENTED PEOPLE BECAUSE FRANKLY IT WILL ALWAYS BE HARD FOR THE GOVERNMENT TO COMPETE WITH THE PRIVATE SECTOR IN THE MOST CUTTING EDGE AREAS, SO WE LOOK FOR YOUR SUPPORT OF THAT KIND OF OF, FRANKLY, PEOPLE COMING FROM YOUR ORGANIZATIONS AND OTHERS TO DO STINTS IN THE GOVERNMENT TO HELP US, YOU KNOW, NOT JUST DO IT HERE AT COMMERCE, BUT FRANKLY HELP LEID THE REST OF THE -- LEAD THE REST OF THE FEDERAL GOVERNMENT. IN TERMS OF BETTER MEASURING CUSTOMER METRICS, DEVELOPING CUSTOMER METRICS, WE'RE CONTINUING TO LOOK AT OPTIONS. MANY OF OUR DATASETS ARE AVAILABLE IN BULK, AND WE DON'T HAVE VISIBILITY INTO HOW THE DATA IS BEING USED AFTER IT'S ACCESSED. WE'VE ASKED OUR LEADERSHIP TO FOCUS ON MEASUREMENT OF USAGE OF OUR DATA IN THE COMING MONTHS. WE DON'T HAVE A SOLUTION FOR THAT. SO IF HELPFUL. YOU PROPOSED A LABS CONCEPT. RIGHT NOW OUR WORK IS VERY MUCH DRIVEN BY THE DEMAND OF THE BUREAUS BECAUSE THEY'RE OUR FUNDERS, BUT WE WILL EXPLORE POSSIBILITIES AROUND, YOU KNOW, SORT OF CDS DOING ITS OWN RESEARCH, THE ABILITY FOR NTIS TO DO THESE PARTNERSHIPS IS AN OPPORTUNITY MAYBE TO DO SOME OF THE RESEARCH THAT YOU WOULD LIKE CONCEPT. FINALLY, YOU KNOW, SEEING THE CDAC GROW AND CONTINUE TO IS ABSOLUTELY CRITICAL, SO YOUR WILLINGNESS EITHER TO SERVE OR TO TELL OTHERS THAT THIS IS A VALUABLE WAY TO SPEND TIME FOR A COUPLE OF YEARS WOULD BE VERY HELPFUL TO US. YOU'VE BEEN A TREMENDOUS RESOURCE, AND I COMMIT THAT OUR TEAM, JUSTIN, BURTON, JEFF, AUSTIN AND OTHERS WILL CONTINUE TO WORK WITH YOU. I WANT TO CLOSE BY SAYING, THOUGH, THE PERSON WHO ACTUALLY MAKES THIS HAPPEN EVERY TIME YOU MEET IS TANYA WHITE. I DON'T KNOW, WHERE IS TANYA? (APPLAUSE) . TANYA IS REALLY THE GLUE THAT PULLS THESE MEETINGS TOGETHER, THAT SUPPORTS NOT ONLY THE COMMERCE DATA SERVICE BUT THE CDAC, AND SO THANK YOU, TANYA, FOR ALL OF YOUR GREAT WORK. WITH THAT, YOU'RE IN GREAT HANDS APPRECIATE EVERYTHING THAT YOU'VE DONE. YOU'VE REALLY HELPED US PUT COMMERCE'S DATA FRONT AND CENTER IN THE OPEN DATA INITIATIVE, AND OUR EFFORT IS FRONT AND CENTER. SO THANK YOU VERY MUCH. >> THANK YOU, SECRETARY. THANK YOU VERY MUCH. (APPLAUSE) . L WE'RE GOING TAKE A SHORT BREAK AT ABOUT 10:05. (SHORT BREAK) . >> (STANDING BY FOR THE MEETING TO RESUME). (PLEASE STAND BY, THE MEETING WILL RESUME SHORTLY) >> WE'LL COME BACK TO THE TABLE, WE NEED TO GET ROLLING, WE'RE A LITTLE BIT LATE NOW. ARE >> (THE MEETING WILL RESUME SHORTLY). >> OH, WE'RE QUIET, EXCELLENT. WE'VE GOT THE LIGHTNING ROUNDS COMING UP. THESE ARE SHORT SESSIONS, SIX SHORT SESSIONS THAT WE ARE GOING TO REALLY HAVE TO PUSH THROUGH QUICKLY. SO THE SPEAKERS NEED TO COVER THEIR STUFF IN A SHORT PERIOD OF TIME AND WE'RE NOT GOING TO HAVE A WHOLE LOT OF TIME FOR DISCUSSION DISCUSSION, BUT THE IDEA IS THAT WE SHOW SOME OF THE BREADTH OF THE WORK BOTH WITH THE COMMERCE DATA SERVICE AND WHAT'S GOING ON IN PTO AND NOAA. LET ME FIRST TURN IT OVER TO JUSTIN.. THANKS AGAIN, BURTON. I WANTED TO MAKE ONE POINT BEFORE WE GO INTO THE LIGHT ANYTHING ROUNDS, AND FIRST, YOU'LL NOTICE THAT IA IAN CALIN HAS ACTUALLY BEEN ON PERSONAL LEAVE SINCE THE LAST CDAC MEETING, SO I WANT TO GIVE HIM A CHANCE TO TALK TO THE GROUP FOR A MINUTE OR SO, IF YOU WOULDN'T MIND. BUT I ALSO WANT WANTED TO RECOGNIZE AS WELL JEFF REALLY HAVE BEEN LEADING THE COMMERCE DATA SERVICE WHILE IAN HAS BEEN ON LEEVMENT BUT IAN, THANK YOU FOR EVERYTHING YOU'VE DONE. YOU'VE OBVIOUSLY BEEN THE VISIONARY BEHIND A LOT OF WHAT WE'RE TALKING ABOUT HERE TODAY AND I WANT TO GIVE YOU A CHANCE TO TALK TO THE GROUP, NOTWITHSTANDING THAT WE'LL FORGIVE YOU SINCE YOU'VE BEEN OUT THE LAST COUPLE MONTHS. NEW SPEAKER: THANK YOU VERY MUCH. I'LL CALL IT PERSONAL CALL IT BABY LEAVE AND TAKING THAT ESSENTIAL FAMILY TIME FOR MY BABY GIRL, VERY EXCITED TO BE BACK HERE AT COMMERCE, ONLY BACK MANY OTHER EXTEND PERIODS OF LEAVE, I HAVE A LOT TO CATCH UP ON BECAUSE I'M LEARNING QUITE A BIT. I WILL SAY VERY BRIEFLY HOW EXCITED I AM THAT SO MUCH OF THE WORK THAT YOU ALL HAVE GUIDED US ON AND SPURRED US TO CREATE WILL CONTINUE, THROUGH ANY INDIVIDUAL'S TIME AWAY, BECAUSE OF THE PRINCIPLES OF LEADERSHIP THAT YOU ALL HAVE CELEBRATED. ONE OF THE BEST WAYS TO LEAD DMAING ANY LARGE ORGANIZATION IS TO FIND THE INTERNAL CHAMPIONS, THE FOLKS THAT HAVE BEEN FOCUS ODD IT LONG BEFORE ANY OF US HAVE SHOWED UP AND IT'S WITH TREMENDOUS PRIDE, GREAT GRATITUDE, JUST A REAL DEEP CONNECTION THAT I WANT TO SAY A FEW VERY SPECIAL THANK YOU'S NOT JUST TO THE FOLKS WHO MADE THIS POSSIBLE BUT THOSE THAT WILL CONTINUE ON THROUGH THE TRANSITION. I'LL START WITH AUSTIN, A GOOD FRIEND WHO INTERVIEWED ME ON DAY ONE, BROUGHT ME IN AND HELPED ME NOT GET MY HEAD CHOPPED OFF AS I WAS BURTON RICE JUST HAS SO MUCH -- BURTON REIST HAS SO MUCH. CR TANYA WHITE IS THE GLUE THAT HOLDS US TOGETHER AND MAKES SURE EVERYONE GETS PAID AND SHOWS UP ON TIME. AND JOHN, THE LEADER OF THE CENSUS BUREAU, YOUR SUPPORT IN SPIRIT, IN FINANCIAL SUPPORT AND DATA HAS BEEN ABSOLUTELY INVALUABLE, WE WOULDN'T BE HERE WITHOUT YOU. MEMBERS OF THE TEAM, WE HAVE NATASSJA, TOM BEECH WHO YOU'LL BE HEARING FROM IN A SECOND, HERARDO, BEHIND ME, THE FOLKS THAT ARE THE FOUNDATION OF THIS DATA THING, REGARDLESS OF THE POLITICAL LEADERSHIP. LISA WOLFFISH I SEE MEMBERS TO THE COMMERCE DATA SERVICE, ED KEARNS HELPING US TO CONNECTIONS BETWEEN THE DEPARTMENT AND NOAA, I'M PROBABLY MISSING SOME FOLKS JUST IN LINE OF SIGHT, BUT I JUST WANT TO SAY THESE ARE THE FOLKS THAT MADE ALL OF THIS POSSIBLE. I'M VERY GRATEFUL FOR ALL OF YOU TO TAKE THIS FANTASTIC INITIATIVE IN MY ABSENCE AND AM VERY GRATEFUL AS WELL FOR YOUR CONTINUED LEADERSHIP AS IT CONTINUES ON INTO THE FUTURE. THE LAST PERSON I'LL THANK PERHAPS IN MANY OTHER WAYS THE MOST IMPORTANT PERSON IS YOUR PARTNER, MY PARTNER, JEFF CHEN, THE DEPUTY WHO WILL BE LEADING THIS ON IN MY ABSENCE, JEFF , YOU KNOW, WHEN WE FIRST GOT INTRODUCED TO THE PRESIDENTIAL IN THE FOAL LOWS PROGRAM, I HEARD ABOUT YOUR WORK AND YOUR BRILLIANCE WAS ALREADY LEGENDARY NOW BE TRANSLATED TO ORGANIZATIONAL BRILLIANCE AND TAKE THIS FANTASTIC INITIATIVE INTO THE FUTURE SOVMENT A SPECIAL, SPECIAL THANKS TO JEFF. NEW SPEAKER: ALL RIGHT W THAT, LIGHTNING PRESENTATIONS. WE'RE TRYING TO DO SHORT 15-MINUTE PRESENTATIONS JUST SO EVERYBODY IS LEVEL SET, WE HAVE PREBRIEFED ONE OR TWO MEMBERS. CDAC SO THAT YOU HAVE A DEEPER DIVE, THESE ARE ONLY 15 MINUTES LONG, WE WANTED TO GIVE THE REST OF THE MEMBERS A SENSE OF WHAT'S GOING ON IN SOME OF OUR SPECIFIC MORNING. THIS IS NOT THE LAST TIME THAT YOU CAN GIVE US FEEDBACK, OBVIOUSLY, IF THERE ARE OTHER INPUTS OR YOU WANT MORE, WE CAN SET UP ADDITIONAL BRIEFINGS, SO DON'T FEEL LIKE YOU'RE LIMITED TO 15 MINUTES. THE FIRST ONE IS ACTUALLY TWO SEPARATE PROJECTS THAT I MENTIONED THIS MORNING THAT WE ARE VERY, VERY PROUD ABOUT DRIVEN BY BEA AND THE COMMERCE DATA SERVICE, WE HAVE LEADERS HERE. SALLY, I'LL TURN IT OVER TO YOU. NEW SPEAKER: THANK YOU FOR HAVING US PRESENT. I'M JOINED HERE BY SALLY THOMPSON, DEPUTY DIRECTOR OF THE BUREAU OF ECONOMIC ANALYSIS, AND WE WOULD LIKE TO GIVE YOU A LITTLE OVER OVERVIEW OF TWO PRODUCTS THAT ARE -- ONE IS CURRENTLY IN DEVELOPMENT, ONE HAS ALREADY BEEN RELEASED, THEY BOTH FOCUS ON ECONOMIC DATA, AND I'LL TURN IT OVER TO SAL LI'L OVER HERE TO HERE, SUCH AN IMPRESSIVE GROUP, I'M A BIG FAN OF ALL YOUR RESUME RESUMES. I JUST WANT TO SAY A LITTLE BIT ABOUT BEA. WE ARE THE AGENCY THAT PRODUCES A LOT OF ECONOMIC STATISTICAL DATA, FOR INSTANCE, THIS MORNING WE RELEASED AT 8:30 THE ADVANCED GD GDP RELEASE FOR THE THIRD QUARTER, AND RIGHT NOW THOSE DATA ARE IN OUR EI AND RIGHT NOW THOSE DATA ARE AVAILABLE THROUGH THE BEAR SYSTEM THAT'S AVAILABLE AVAILABLE, THAT JEFF WILL DEMONSTRATE IN A MINUTE. BUT BA HAS A BIG COMMITMENT TO MAKING DATA AVAILABLE. WE'VE GOT 12 AP 12 API'S WITH THOUSANDS OF DATA SERIES IN THEM, WE HAVE A NEW -- INTERFACE WHERE YOU CAN PULL OUT PIECES OF DATA. WE'VE GOT ALL SORTS OF OTHER WAYS OF SERVING UP DATA. BUT NOW WITH THE API, THE ARTICLES, WE HAVE EVEN MORE. USERS, EXISTING USERS, I'LL TELL YOU WHEN WE STARTED WORKING WITH R, I HAVE TO ADMIT, LOOK AT MY HAIR, IT WAS NOT SOMETHING THAT I'VE BEEN USING EVERY DAY, BUT I CALLED MY 29-YEAR-OLD SON WHO IS AN ASSISTANT PROFESSOR AT THE UNIVERSITY OF FLORIDA AND SAID, WELL, IS THIS SOMETHING GOOD? HE SAID OH, YEAH, YOU NEED TO WORK ON THAT. SO HE AND OTHER ANALYSTS IN ACADEMIA, IN THE PRIVATE SECTOR, IN GOVERNMENT INCREASINGLY ARE GOING TO BE LOOKING FOR NEW TOOLS, THE TOOLS THEY USE, THE OPEN SOURCE ONES THEY'VE BECOME USED TO AND WE'RE JUST AN EXAMPLE THAT I REALLY APPRECIATE THE COMMERCE DATA SERVICE AND ESA PROVIDING A LITTLE NUDGE TO US IN THAT AREA, ON AND ARE PARTNERING HAPPILY. OKAY. SO THE BEAR LIBRARY OR BEAR SOME OF YOU LIKE TO CALL IT IT, WE HAPPEN TO SAY BEA JUST NATURALLY, BUT ANYWAY, FOR API, IS AVAILABLE ON GITHUB, WHAT'S AVAILABLE THERE NOW ARE ALL OUR NATIONAL AND PRIVATE ACCOUNT DATA AND DETAIL, IF YOU WANT TO KNOW MORE ABOUT THAT, I CAN ANSWER, AND OUR REGIONAL DATA. SO LIKE I SAID, THE DATA RELEASED THIS MORNING ARE AVAILABLE RIGHT NOW THROUGH THIS SYSTEM. JEFF? JEFF CHEN: I THINK THE NIFTY THING IS THAT WE'VE BEEN SEEING EMERGING DEMAND SEGMENT THAT'S REALLY USING OPEN SOURCE TOOLS. IF YOU LOOK AT UNDERGRADUATES AND GRADUATE STUDENTS ACROSS THE COUNTRY, THEY TURN TO OPEN SOURCE TOOLS TO DO THEIR ANALYSIS ANALYSES, THERE'S TIME SERIES FORECAST, YOU CAN PROBABLY FIND SOMEBODY USING OPEN SOURCE. DATA JOURNALISTS ARE VERY MUCH USING THIS AND WE HAD AN AWESOME WHERE THEY WERE INTERESTED IN USING OUR SYSTEM. IF YOU THINK ABOUT THE BEA A PI OR ANY API, YOU MAY HAVE TO TAKE TIME TO GET WHAT'S IN IT, SO THIS LIBRARY IS REALLY THAT WITH JUST A LITTLE BIT, YOU CAN GET RIGHT INTO THE DATA AND START BUILDING YOUR PRODUCTS. SO I'M GOING TO DO -- THIS WORK, YOU CAN SEE, THIS IS THE R COUNCIL WITH THE R SCRIPT ON THE LINES OF CODE HERE. SO ALL I HAVE TO DO IS OPEN THE LIBRARY, FORTUNATELY MY API KEY IS SHOWING AWAY AFTER THIS, SO THEN CAN SEARCH THE API AND IT WILL KICK BACK A RESULT, KIND OF CRAZY INITIALLY, BUT WHAT IF I WANT TO SEE A NICE CONVENIENT VERSION OF IT? SO I CAN RUN A B BEA SEARCH QUERY WHERE I SPECIFY BACK A JAVASCRIPT INTEGRATED TABLE WITH RESULTS. THEN IF YOU KEYWORDS, YOU CAN SEARCH AROUND THE TABLE AND GET ALL THE RIGHT INFORMATION. THE COOL THING IS THAT YOU ONLY NEED A FEW PIECES OF INFORMATION TO BE ABLE TO EXTRACT DATASET FROM THE API. SO YOU SPECIFY A LIST OF FEATURES, SOME OF THESE THINGS CAN ACTUAL BE TURNED INTO DEFAULT IN LATER VERSIONS, AND THEN SO WE RUN THIS, AND THEN IF I JUST P THE DATA -- IF I JUST WANT THE DATA, WE CAN HIT API, THEN WE HAVE A PAYLOAD. PAYLOAD WILL COME BACK WITH DATA THAT KIND OF LOOKS LIKE WHEN YOU'RE GOING TO THE TABLES, YOU CAN SEE THE RESULTS GOING ACROSS THE TIME VARIABLE GOING ACROSS THE TOP, BUT WE CAN RESHAPE THE DATA IN ANY NUMBER OF WAYS SO THAT IT'S IN A LONG FORMAT, SO IF YOU'RE TO LOOK AT BEA LONG, WE CAN A NICE CLEAN SET OF DATA THAT CAN BE READILY USED. THEN IF WE REALLY, REALLY TO WANT GET HAVE A LITTLE VISUALIZATION LIBRARY THAT WILL TAKE THE DATA AND IT WILL VISUALIZE IT FOR YOU YOU. SO FOR THOSE WHO ARE NOT ENTIRELY FAMILIAR WITH THE DATA AND WANT TO KNOW WHAT'S IN THAT DATA, YOU CAN EXPLORE INFORMATION IN A MORE USABLE, SIMPLE WAY. SO THAT JUST IS SOME OF THE DEMAND SEGMENTS, BUT BEYOND THE INITIAL BEA API. SO WE HAVE THIS OPEN DATA PARTNERSHIP AND IT'S AN EFFORT BETWEEN COMMERCE BE A ON ONE SIDE OF THE POND AND THE EUROPEAN COMMISSION IS EUROSTAT AND CONNECT ON THE OTHER SIDE OF THE POND, AND THE EFFORT IS REALLY FOCUSED ON UNDERSTANDING HOW TO DEVELOP A HARMONIZED PROTOTYPE OF EU-US DATA, AND IT REALLY STARTED OFF WITH SALLY AND BE ECONOMISTS HAVING A NICE CONVERSATION WITH EUROSTAT ECONOMISTS TO FIGURE OUT WHERE IS THE COMMON GROUND IN ECONOMIC DATA, AND ONCE WE FIGURED OUT WHAT THAT COMMON GROUND IS, TO SHOW A THING, WE LIKE TO USE THAT TERM, WE ARE CURRENTLY BUILDING A LIBRARY USING OPEN DATA LINK TECHNOLOGIES WITH A FRONT END TO QUERY AGAINST THE DATA FROM BOTH API'S SO WE CAN HAVE A HEALTH CARE REFORM ONIZED VIEW OF AN INITIAL CURATEED SET OF DATA. SALLY: THAT LIBRARY WILL INCLUDE DATA ON THE STATE AND METROPOLITAN BASIS, GDP DATA LEVELS FROM THE EUROPEAN COMMUNITY WHICH , WHICH STANDS FOR NOTIONAL UNITS OF TERRITORIAL SOMETHING, BUT THAT'S A COMMON TERMINOLOGY IN EUROPE, AND WE'VE GOT THINGS WORKED OUT IN TERMS OF MAKING SURE WE HAVE COMPARABLE DATA. I THINK THERE WILL BE LOTS OF DOCUMENTATION ABOUT HOW THE DATA SOURCED AND WHAT HAVE YOU, BUT WE'LL SAY RESEARCH IS A LOT OF OF -- RESEARCHERS, A LOT OF WORK WHO WANT TO DO COMPARISONS OF STATE ECONOMIES IN THE U.S. OR METROPOLITAN COMMITTEE ECONOMY IES IN THE U.S. WITH SIMILAR ECONOMIES IN EUROPE, GOOD FOR PROVIDING INFORMATION ON LOOKING AT NEW BUSINESS OPPORTUNITIES, NEW EXPANSION, NEW INVESTMENT, AND WE THINK THERE'S GOING TO BE GOOD USE OF THIS NEW LIBRARY. JEFF CHEN: AND TOGETHER WITH ALL OF THESE DIFFERENT PIECES, THE HOPE IS THAT WE CAN FOSTER GREAT COOPERATION AND BETTER UNDERSTANDING WHEN WE ARE GOING INTO JUST EVEN LIKE RESEARCH OR ECONOMIC TALKS SO THAFER ONE IS LOOKING AT -- SO THAT EVERYONE IS LOOKING AT MORE INFORMATION. POTENTIAL FROM JUST BUILDING OUR LIBRARY BECAUSE IT'S A LITTLE BIT EXTREME, BUT IT CAN HELP INFLUENCE A LOT OF THINGS DOWNSTREAM. NEW SPEAKER: WE CAN DO IT VERY QUICKLY AND OF COURSE THERE WILL BE TIME FOR MORE DISCUSSION. APPROXIMATE NEW SPEAKER: CONGRATULATIONS TO THOSE. I ESPECIALLY LIKE TO SEE THAT YOU'RE NOW COLLABORATEING THIS EFFORT IN COUNTRIES, IN THIS CASE THE EUROPEAN COMMISSION AND REALLY INTER INTERLINKING EUROSTAT DATA AND U.S. DATA BECAUSE ULTIMATELY SCIENCE, TECHNOLOGY AND INNOVATION IS THE GLOBAL EFFORT AND AS YOU KNOW, GLOBAL CIRCULATIONS ARE GOING ON AND IT'S EXTREMELY IMPORTANT TO UNDERSTAND WHERE THESE GO, WHERE ET CETERA, AND IT'S ONLY POSSIBLE IF YOU CONNECT THESE DIFFERENT DATA SILOS THAT YOU GET A MORE HOLISTIC UNDERSTANDING OF THIS. ULTIMATELY, I ALREADY MENTIONED THIS, YOU MIGHT ALSO WANT TO MAKE CLAP RAITIONZ WITH OACD AND ALREADY HAVE 34-PLUS MEMBER COUNTRIES BUT I THINK THIS IS A WONDERFUL STEP AND MAYBE ONE MORE COMMENT IS TO REALLY BRING THESE NEW DATASETS TO THE COMMERCE DATA ACADEMY, WHICH FROM WHAT I UNDERSTAND ALL OF THOSE VIDEOS ARE FREE AND OPEN TO USE. FINALLY, ADVERTISE TO TEACHERS NOT ONLY IN GOVERNMENT BUT ALSO IN THE ACADEMIC SETTING AND GET RESEARCHERS INTERESTED IN ADDITION TO GETTING INDUSTRY INTERESTED BECAUSE THEN THESE TEACHERS CAN TEACH THE NEXT GENERATION THAT THIS IS NOW AVAILABLE AND CAN BE USED, AND THIS WAY STUDENTS GO OUT INTO INDUSTRY AND THEY ALREADY KNOW THAT THIS EXISTS AND DHOANLT HAVE TO REPLICATE THIS EFFORT, BUT THEY CAN NOW USE ALL THESE NEW INTERLINKED DATA SOURCES. SALLY: THANK YOU. A COMMENT, SOME OF OUR EARLY TESTERS HAVE BEEN ACADEMIC USERS, SO WE'VE MODIFIED OUR INSTRUCTIONS AND WHAT HAVE YOU TO MAKE IT AS USER FRIENDLY AS POSSIBLE. NEW SPEAKER: VERY GOOD. NEW SPEAKER: I KNOW WE'RE SHORT ON TIME OF THE BURTON -- DOES ANYBODY WANT TO SPEAK? NEW SPEAKER: I THINK THIS LOOKS GREAT. I WOULD SUGGEST, IT LOOKS LIKE ON THE GITHUB PAGE YOU REFERENCE API PAGE, WHICH IS GREAT, IT LOOK LIKE API DOESN'T GIVE TROARCHES BEAR PACKAGE, YOU KNOW IF THEY WANT TO USE PA PI, THERE'S AN R TOOL ALREADY WRITTEN FOR IT. ALSO IT LOOKS LIKE THE PACKAGE IS NOT ON CRAYON, IT WAS REMOVED FROM CRAYON. JEFF CHEN: THAT'S IN THE PROCESS. NEW SPEAKER: YOU DEFINITELY WANT IT ON CRAYON OBVIOUSLY OR PEOPLE AREN'T GOING TO FIND IT. THEN PART OF THE EU PARTNERSHIP WAS ALSO SEARCH CAPABILITY, IS THAT RIGHT, SO YOU COULD FIND SEARCH, YOU COULD FIND DATASETS MORE EASILY, IS THAT RIGHT? JEFF CHEN: THAT'S THE GOAL. NEW SPEAKER: WOULD THAT LOOK A LOT LIKE DATA.GOV SEARCH FUNCTIONALITY? >> ?. JEFF CHEN CHEN: IT WOULD BE IN CONSOLE EXPERIENCE FOR OUR USERS, INITIALLY TO CURATE A SET OF COMPARABLE DATASETS. NEW SPEAKER: SO IT'S IN CONSOLE IN R, IT'S A USER, IT'S NOT WEB. JEFF CHEN: NOT QUITE. NEW SPEAKER: OKAY. GREAT. NEW SPEAKER: MAYBE I COULD PUT A TOPPER FROM MY PERSPECTIVE, WHICH IS THIS IS ONE OF THOSE STORIES THAT YOU COULD TELL THAT ENTIRE DATA EFFORT BECAUSE IT STARTED AS A PROJECT THAT ONE OF THROUGH THE ACADEMY, ANDREA CHOA CHOACA WORKED DIRECT WELL OUR FB FBEA, WORKED DIRECTLY WITH OUR TEAM AT CDS, THEY DROVE IT FROM THE BUREAUS, IT'S SEATED AT B HE A, OWNED IT AND REALLY MADE IT PART OF THEIR CORE MISSION ON OPEN DATA, OPEN SOURCE, ITS SEED SEED, SO WE START WITH SOMETHING FUNCTIONALITY, WHICH IS EXCITING EXCITING, JEFF AND WHEN WE PRESENT IT HAD TO EDITORIAL BOARDS AND THE FOLKS WHO NEED TO THAT AND WE'RE GOING TO LOOK TO DRIVE USAGE OF IT, YOU SEE LIGHTS GOING OFF BECAUSE THAT'S EXACTLY WHAT THEY NEED TO BE ABLE TO USE AND VISUALIZE OUR DATA OVER THE LONGER TERM, AND I WILLING TO PUT SOMETHING OUT THERE THAT HAS GLITCHES, RIGHT, THAT YOU HAVE TO START, RE RELEECHES THE PRODUCT AND GET SOME FEEDBACK, THAT'S REALLY IMPORTANT. THE EU-US PROJECT SHOWS WHAT YOU CAN DO WHEN YOU TURN THE DATA PILLAR TOWARDS TO ALIGN IT WITH OUR POLICY GOALS, AND I MET, BECAUSE I'VE BEEN LEADING SOME INTERNATIONAL DIALOGUES, WITH THE EUROPEAN COMMISSION, I MET WITH THE HEAD OF DG CEK RS ROBERTA VIOLA, WE AGREED TO TRY TO GET THIS PROJECT DONE THIS SUMMER. THINK JULY, WE ASKED OUR TEAMS TO GET TOGETHER AND SEE WHAT YOU CAN ACCOMPLISH ON AN OPEN SOURCE, OPEN DATA PROJECT, SO JUNE, JULY JULY, TO NOVEMBER TO RELEASE THE ALPHA VERSION IN OPEN SOURCE WITH THE IDEA OF OPEN DATA, THAT'S SOMETHING THAT A COUPLE YEARS AGO NONE OF US WOULD HAVE THOUGHT WAS POSSIBLE BUT FOLKS LIKE SALLY AND JEFF WHO ARE REALLY PUTTING THEIR LEANING SHOULDERS IN TO GET SOMETHING LIKE THIS DONE. SO I'M PROUD OF THIS, BUT IT ALSO SHOWS POLICY ALIGNMENT WITH OUR ARE DATA PILLAR. NEW SPEAKER: JEFF AGAIN TO DISCUSS SEARCH STREAM ANALYSIS. NEW SPEAKER: A BIT OF A THEME. JEFF CHEN: A BIT OF A TRIATHLON FOR PRESENTATIONS PRESENTATIONS, BUT DOUBLE HEADER PULL UP THE OTHER PRESENTATION. PRESENTATION FOCUSES ON A SEARCH STREAM PROJECT THAT WAS DEVELOPED BY DATA SCIENTIST STARTING ON MY TEAM, AND IT WAS DONE IN CONJUNCTION WITH THE U.S. PATENT AND TRADEMARK OFFICE OFFICE, AND TOM BEECH HAS BEEN A WORK. SO EVERY DAY, THE PATENT STANDARDS -- WILL THERE ARE MANY OTHER PIECES, DEPENDING ON WHICH ABOUT. BUT WHAT IF ONE OF THOSE PIECES WERE MISSING? LIKE WHAT IF YOU DIDN'T HAVE A WAY TO CONNECT ALL THE STANDARD TO NON NONSTANDARD TERMS SO THAT YOU HAVE A CONSISTENT SEARCH? THERE ARE A LOT OF DIFFERENT THINGS THAT HAPPENED IN SEARCH THAT MAY WITH THE PATENT OFFICE, THEY'RE WORKING ON THE NEXT GENERATION SEARCH ENGINE, IT'S A MAG ANY MAGNIFICENT PIECE OF WORK, AND FOR THEIR SEARCH THEY'RE USING SOLAR, BUT TO GET BEYOND THAT, YOU NEED TO HAVE A GOOD SYNONYM ENGINE. IF YOU DON'T HAVE A SYNONYM ENGINE, YOU RUN INTO ISSUES. IF YOUR RECORD HAS CERTAIN TAGS ON IT, IT MIGHT NOT GET UPDATED ALL THE TIME, SO THEN THAT RECORD IS ONLY AS GOES GOOD AS THE TAKS, AND IF YOU DON'T HAVE THE SYNONYM ENGINE TO CONNECT ALL THE DIFFERENT TERMS, STAR SAYS THERE'S THIS THING CALLED A -- WHAT IS THIS ACRONYM ACRONYM? WITHOUT A SYNONYM ENGINE YOU CAN'T REALLY FIGURE OUT WHAT AN ACRONYM IS EFFECTIVELY. TERMINOLOGY IS POKEMON, WHETHER IT'S THE TERMINOLOGY AND ACTUAL DOCUMENTATION, THE ACTUAL ARTICLES OR ENTRIES IN YOUR INDEX OR THE TERMS THAT ARE ENTERED IN HAVE I ON PATENT EXAM EXAMERS, SO IF YOU DON'T HAVE IT IT, YOU CAN'T CATCH THEM ALL OR MATCH THEM ALL REALLY. LASTLY, THE DISES AM BIG YAITION. THE WORD CHRISTMAS TREE FOR MANY PEOPLE MEANS THE CHRISTMAS TREE, INDUSTRY IT MEANS SOMETHING VERY DIFFERENT. SO AS PART OF THE GREATER TECHNOLOGY ROAD MAP, THE DEVELOP A NEW RANK ENGINE FOR DIFFERENTIATED TERMS, WE CAN SOLVE THE DISAMBIGUATION PROBLEM PROBLEM, WITA PROBLEM, POKEMON PROBLEM, YOU NAME IT, WE'RE WORKING TOWARDS IT. SO TO GET AT THIS, PTO PROVIDED US WITH JUST OVER ONE YEAR'S WORTH OF SEARCH LOGS, ABOUT 89 MILLION SEARCH LOG RECORDS, WHICH MIGHT BE A BIT FOREGOING, MIGHT NOT BE THAT MUCH FOR INDUSTRY, BUT IT'S A GOOD START. AND FROM THE LOGS WE CAN HAVE THIS RICH HISTORY OF ALL THE PATENT EXAMINERS' SEARCH TERMS THAT THEY PUT IN IN THE PAST, AND THIS WAY YOU CAN GET GET -- YOU HAVE THIS COLLECTIVE KNOWLEDGE OF WHAT IS A CONSISTENT TERM. SO WE STARTED TO DEVELOP THIS PYTHON PROGRAM THAT ESSENTIALLY DOES ALL THE ET ETL ON THE SEARCH LOGS SO THAT WE CAN UNDERSTAND WHAT ARE THE WORDS THAT ARE RELATED AND THEN ESTIMATE THE RANKED LINEAR CONDITIONAL PROBABILITY ESTIMATES BY OUR AREA. SO THIS WAY YOU HAVE A REALLY HONEED IN SET OF RECOMMENDED TERMS PER ART AREA. AND THIS EFFORT IS VERY MEANT TO FIT INTO SOMETHING LIKE HAVE THE CAPABILITY OF RECOMMENDING ALL THE OTHER TERMS INTO YOUR SEARCH SO THAT YOU CAN GET TO THE RIGHT PIECE OF PRIOR ART MORE CONSISTENTLY, AND IT'S PART OF THIS OVERALL THEME OF CONSISTENT SEARCH AND CONSISTENT ONE SYNONYM ENGINE WILL BE ABLE TO REACH OVER 8,000USPTO EXAMINERS, HIT 600,000 APPLICATIONS, EACH APPLICATION TAKES ABOUT FIVE HOURS ON AVERAGE, AND IT FACILITATES A MORE CONSISTENT WAY OF DOING BEHIND THE SCENES WORK. IT'S PRETTY FUN TO SEE HOW THIS STUFF EVOLVES. IT'S OUR LITTLE PART IN HELPING THE PTO, AND I THINK IT'S REALLY JUST THE BEGINNING. THANK YOU. SPEAKER: THAT WAS TERRIFIC. DEAN AND COLIN ARE OUR DISCUSSANTS ON THIS. VADIM: WHAT I REALLY LIKE IN THIS PARTICULAR EXAMPLE IS LET'S START WITH THE SEARCH AND FOR ALL THE PEOPLE WHO SEARCH FOR US US, IT'S A DONE DEAL, IT WAS SOLVED TEN YEARS AGO, BUT THIS PARTICULAR TYPE OF SEARCH VERY, VERY DIFFERENT THAN WHAT EVERYBODY ELSE THINKS OF A SEARCH, AND WHAT THE PATENT OFFICE IS VERY DIFFERENT THAN WHEN YOU GO TO GOOGLE AND TRY ON IT'S VERY WELL DEFINED WITHIN WHAT THE OFFICE IS. SO THEY START LOOKING AT THE PROBLEM AND IDENTIFIES THE PROBLEM, IDENTIFIES WHAT CAN AND CANNOT DO, SOLAR IS OPEN SOURCE, SEARCH ENGINE, HAS LIMITATIONS, AND IT WAS BUILT FOR SOMETHING ELSE BUT AND BUILT A SOURCE WHICH CAN WORK TOGETHER WITH SOLAR OR TECHNOLOGY WHICH WAS GREAT BUT IT SOLVES THE PROBLEM WHICH THE PATENT OFFICE HAS, AND THE SOLUTION THEY CREATED IS NOT A ONE-TIME SOLUTION. IT GOES THROUGH, CREATE WHATEVER MODEL, PUT IT OUT AND THEN RUN T BUT IT'S BUILT IN A WAY THAT IT WILL WILL BE USED BY SOME SOME OTHER INDUSTRY, I DON'T KNOW. THEY SHOULD BE ABLE TO PICK IT UP AND IS CONTINUALLY REFINED AND GOING THROUGH. HOPEFULLY IT'S A PROJECT THEY BUILD IT ONCE AND IT WILL RUN FOR A NUMBER OF YEARS AND UPDATE ITSELF AND WILL PATENT OFFICE WILL USE GOING FORWARD. SPEAKER: I THOUGHT IT DIFFERENT DIRECTION, BECAUSE WHAT I THINK YOU'VE CREATED IS AN ONTOLOGY FOR ANOTHER INDUSTRY INDUSTRY, SO FOR INSTANCE, I'M IN THE INDUSTRIAL INDUSTRY, SO WHEN I LOOK AT CHRISTMAS TREES, THEY TIE TO OIL AND GAS FOR SHARE. AIRCRAFTS, WHEN YOU TALK SOMETHING YOU COVER OVER A BODY, IT'S SOMETHING THAT WRAPS AROUND A PLANE ENGINE. IN THE INDUSTRIAL INDUSTRYIES RIGHT NOW , HAVING THAT DATABASE IN WHICH YOU ARE LEARNING NEW TERMS AND CONNECTING THE NEW TERMS TOGETHER IS VASTLY IMPORTANT. THE REASON WHY WATSON DID SO WELL IS THEY HAD WIKIPEDIA WHICH WAS AN ENGLISH DATABASE. THIS IS ONE YOU'RE CREATING FOR AN INDUSTRY, SO THE OIL AND GAS INDUSTRY WOULD APPRECIATE IT, THE AIRCRAFT INDUSTRY WOULD APPRECIATE T BECAUSE YOU'RE GOING THROUGH ALL THE PATENTS, SO ONCE YOU LAY THAT ONTOLOGY OUT, I THINK YOU MAKE THAT, YOU EXPOSE THAT TO THE REST OF THE COMPANIES, AND SAY YOU HAVE AN ONTOLOGY TO ALL THESE TERMS RELATED TO THE PATENT, THEY WOULD LOVE THAT, BECAUSE WE CAN'T FIND HA RIGHT NOW. SO I THINK THIS IS A HUGE THING, JUST GET SEARCH BETTER, SO I CAN TELL YOU I THINK THERE'S A BIGGER BASE HERE FOR WHAT YOU HAVE. EXCELLENT WORK. SPEAKER: THANK YOU. SPEAKER: OTHERS? SPEAKER SPEAKER: ON THIS, OR -- FEEDBACK? SPEAKER: FOR US, I AGREE WITH BOTH, I THINK FOWRMS AND THE CDS IT'S A PER -- FOW AND THE CDS IT'S A PERFECT WAY TO BRAND AND EXPLAIN WHAT OUR DATA SCIENTISTS CAN DO REALLY AND STAR'S EFFORTS WERE TERRIFIC THIS TOGETHER IN A PARTNERSHIP WITH OUR CAREER LEADER TOM AT PT PTO, MICHELLE LEE, OUR UNDER UNDERSECRETARY THERE, HAS REALLY WITH TOM, BUT TO PARTNER IN THAT WAY ALLOWED US TO SHOW OTHER BUREAUS WHAT DATA SCIENCE CAN BRING TO YOUR WORK, RIGHT, AND HOW IT CAN MAKE IT MORE EFFICIENT, WHICH FOR US SCOPING THE NEXT ROUND OF WORK HAS BEEN ABSOLUTELY CRITICAL. GREAT. JEFF, I THINK YOU GET OFF THE PODIUM FOR A FEW MINIMUM IN. WE'RE GOING TO HAVE STEVEN DEVINE FROM THE ECONOMIC DEVELOPMENT ADMINISTRATION AND STEVEN WAS ONE OF OUR INITIAL PARTICIPANTS IN THE DATA ACADEMY DETAIL INTO THE CDS AND WE'RE GOING TO TALK ABOUT ONE OF THE PROJECTS THAT HAS BEEN DRIVEN BY A NEED AT EDA THAT STEVEN AND OUR CDS TEAM REALLY WORKED ON. SPEAKER: THANKS, JUSTIN. HELLO HELLO, MY NAME IS STEVEN DEVINE, ECONOMIC DEVELOPMENT ADMINISTRATION OR EDA, WHICH IS A BUREAU HERE AT DEPARTMENT OF COMMERCE, AND I WAS FORTUNATE ENOUGH TO BE ABLE TO PARTICIPATE IN THE COMMERCE DATA ACADEMY, WHICH I HAVE TO SAY WAS AMAZING LEARNING OPPORTUNITY. I'M THANKFUL TO THE COMMERCE DATA SERVICE FOR PROVIDING IT. SO MY PROJECT THERE WAS CALLED THE EDA GRANT VIEWER, AND IT WAS INTEND INTENDED TO SOLVE A CHALLENGE WE AT EDA, WHICH WAS BASICALLY THAT WE HAVE TWO DISTINCT DATABASES FOR GRANT ADMINISTRATION AND PERFORMANCE DATA, AND THE REASON FOR THAT IS WE'RE CURRENTLY TRANSITIONING OUT OF A LEGACY DATABASE CALLED OPCS, IT'S THE ACRONYM, STANLDZ FOR OPERATIONAL PLANNING AND CONTROL SYSTEM, AND THAT DATES BACK TO THE 1990S AND WE'RE TRANSITIONING OUR PROGRAMS INTO A NEW DATABASE CALLED GRANTS ONLINE, ACRONYM GOL UP THERE, SO THERE'S NO CONSOLIDATED PLACE WHERE ALL THE DATA IS MAPPED UP, HASN'T OCCURRED YELT. SO ANOTHER ISSUE IS THAT OPCS IS NOT THE MOST USER FRIENDLY SYSTEM IN TERMS OF QUEERL RI CAPABILITIES, IT'S BASED ON THE BULLIAN QUERY SYNTAX, WHICH YOU HAVE TO KNOW A LITTLE BIT ABOUT TO USE PROPERLY, AND PART OF THAT DIFFICULTY, A LOT OF OUR STAFF WAS NOT HAVING ACCESS, THEY DIDN'T EVEN HAVE ACCOUNTS TO THE OPCS DATABASE SO THEY COULD NOT DIRECTLY PULL THE DATA THEMSELVES, THEY WOULD HAVE TO REACH OUT TO A COLLEAGUE, ASK FOR AN E-MAIL, ET CETERA, THEN GRANTS ONLINE ALSO HAS A FAIRLY LIMITED QUERY CAPABILITY FOR USER INTRER FACING, IF YOU WANT TO PULL LIKE A TAILORED QUERY FROM THERE, YOU HAVE TO SEND AN E-MAIL TO GRANLTS ONLINE HELP DESK AND THEY'LL PULL THE DATA FOR YOU AND THEN E-MAIL IT BACK TO YOU, SO THAT TAKES A FAIR AMOUNT OF TIME. AND THEN LASTLY LASTLY, NEITHER DATABASE HAD MUCH IN THE WAY OF VISUALIZATION CAPABILITIES, SO THAT WAS A CHALLENGE THAT WE WERE GOING ON TRY TO ADDRESS AND WE FIGURED SINCE IT WAS FAIRLY TIME CONSUME STAFF TO USE THE DATA NOW, IF WE WERE TO CHANGE THAT, YOU KNOW, THEY WOULD ESSENTIALLY USES THE DATA MORE THAN THEY CURRENTLY ARE TO HELP DRAFT DECISIONS. SO HERE IS THIS SCREENSHOT ON THE LEFT IS THE OLD OPCS QUERY INTERFACE USING THE CR BOLIAN SYNTAX, AND ON THE RIGHT SCREEN SHOGHTS FROM WAS A SIMPLER QUERY INTERFACE AND VISION CAPABILITIES. SO THE INCREASE THE DATA USABILITY ACROSS EDA, AND WE DID THAT BY TRYING TO MERGE AS BEST WE COULD THE DATA FIELDS FROM THE OLD OPC OPCS AND GRANTS ONLINE DATABASES INTEGRATEED DATASET AND THEN WE LAID ON TOP OF THAT THE USER INTERFACE THAT STAFF COULD INTERACT WITH. WE DID THIS OPEN R AND SHINY, OUR INITIAL ATTEMPT WAS THIS WAS A MORE IN HOUSE FOR PERFORMANCE TEAM AND REAL IERMINGZED OTHER STAFF COULD BENEFIT FROM IT, WE SCALED UP FROM THAT INITIAL BASE IN R, IT IS BASED ON MODERN JAVA SCRIFT LIBRARIES AND WE TRIED TO IMPLEMENT AJIEL DEVELOPMENT AND USER CENTERED DESIGN. NEW FUNCTIONALITY AT EDA GRANTS BUREAU, YOU CAN SEARCH AND DOWNLOAD AND COMBINE OPCS AND GRANTS ONLINE DATA, YOU CAN BUILD AND SHARE CUSTOM QUERYIES, AND YOU CAN VISUALIZE DATA WITH INTERACTIVE MAPS, CHARTS AND TABLES, AND WE COULD ALSO KEEP TRACK ON USAGE METRICS. SO THE BUREAU IMPACT, IT'S HELPED OUR STAFF HAVE GREATER ACCESS TO THE PROJECT OFFICERS WHO SIT ON INVESTMENT REVIEW COMMITTEES, AND EDA'S INVESTMENTS, BY THE WAY, ARE ECONOMIC DEVELOPMENT AWARDS FOR THINGS LIKE PUBLIC WORKS, ROADS, SUEERS, WATER LINES -- SEWERS, WATER LINES, WORKFORCE TRAINING PROGRAMS, BUSINESS INCOULD YOU BAY TORRES, ASSORTED FEASIBILITY STUDIES AND SUCH. DECIDEING WHAT GRANTS TO AWARD, THEY CAN PULL UP THE MAP AND MAKE SURE THAT THE HISTORICAL DISTRIBUTION HAS BEEN STAFF CAN SELF-SERVE THE DATA THEMSELVES WHEN THEY GET CONGRESSIONAL AND DATA REQUESTS INSTEAD OF HAVING TO REACH OUT TO COLLEAGUES AND WAIT FOR THE DATA TO BE RETURNED TO THEM. I'M NOT SURE WHAT HAPPENED THERE. ALSO EXECUTIVE LEADERSHIP TEAM, THEY TRAVEL OFTEN AROUND THE DROWRNT MEET WITH STATES, -- AROUND THE COUNTRY TO MEET WITH STATES AND CITY OFFICIALS, THIS ALLOWS THEM DIRECT ACCESS TO DATA AND FIND OUT WHAT APPLICATIONS AND AWARDS THEY CAN ADDRESS THEIR CONCERNS IN DETAIL, AND AS WELL, OUR MANAGERS CAN HAVE BETTER VISIBILITY ON THE GRANT WORK FLOW, FOR INSTANCE, YOU KNOW, IDENTIFYING QUICKLY WHAT PUBLIC WORKS GRANTS ARE STILL PENDING OR WHAT PERFORMANCE OUTCOMES HAVE BEEN ACHIEVED DURING THE JOBS CREATED OR PRIVATE INVESTMENT. SO THERE'S BEEN RAPID ADOPTION AT EDA OVERALL, WE'VE GOT ABOUT 50 USERS AND THE FEEDBACK HAS BEEN POSITIVE. THEY'RE SAYING IT'S EASIER TO USE THAN THE PREVIOUS OPCS SYSTEM, AND WE'VE TRIED TO ACHIEVE THE GOAL PROVIDING DATA ACCESS TO A BROADER AUDIENCE. SO A LOT OF FOLKS WHO DID NOT PREVIOUSLY HAVE AN ACCOUNT, THE ABILITY TO DIRECTLY HOLD THE DATA NOW HAVE THIS EDA GRANT DATA INSTALLED AND CAN ACCESS THE DATA, INCLUDING TWO OUR DEPUTY GRANT SYSTEM SECRETARIES. SO OUR NEXT STEPS, BASICALLY WE'LL CONTINUE REFINEING THE GRANTS VIEWER, ADDING MORE DOCUMENTATION AND IMPLEMENTING SOME UPDATES THAT HAVE BEEN RECOMMENDED, WE'VE PROVIDED TRAIN THE STAFF THROUGH AGENCY REB NARS, WE'LL CONTINUE THAT PROCESS -- WEBINARS, WE'LL CONTINUE THAT PROCESS AND WE'LL CONDITION THE APPLICATION FOR JAVASCRIPT AND HTML AND LOOK INTO HOST IT GO ON EDA'S PUBLIC WEBSITE. ALSO WE'VE TALKED WITH SEEM TO HAVE A SIMILAR ISSUE TRANSITIONING OUT OF LEGACY DATABASES TO GRANTS ONLINE SO THEY MIGHT LOOK INTO SIMILAR OPTIONS. THAT CONCLUDES MY BRIEF, PENDING ANY QUESTIONS. SPEAKER: GREAT. KARIN, I KNOW YOU'RE THE DISCUSS ABILITY ON THIS, AND STEVEN, IF YOU WOULDN'T MIND GOING BACK TO THE SLIDE THAT HAS THE BEFORE AND AFTER. CARE RIN: THANK YOU SO MUCH FOR YOUR BRIEFING AND FOR THE WORK YOU'VE DONE WITH THIS BECAUSE I HAVE BEEN ON THE RECEIVING SIDE OF THAT OLD SIDE AND THAT'S NOT WHAT A PROGRAM OFFICER THAT YOU WANT TO BE MANAGING YOUR PORTFOLIO SHOULD HAVE TO TRY TO NAVIGATE, SO IT'S A TREMENDOUS ADVANCEMENT TO GO FROM FROM THAT THINGS THAT YOU SAID JUST DURING YOUR COMMENTS WAS STAFF USE DATA LESS. WELL, WHO WOULD WANT TO USE THAT OLD THING, RIGHT? STAFF USE DATA LESS, WE WANT STAFF TO USE DATA MORE , AND IF YOU GIVE THEM THE KIND OF TOOLS THAT THIS GIVES THEM, THEY CAN DO A LOT MORE WITH THE ENERGY THAT THEY HAVE FOR THE WORK THAT THEY NEED TO DO, INSTEAD OF FIGHTING WITH THAT OLD. SO I REALLY APPRECIATED THE EFFORT THAT'S GONE INTO THIS AND ALSO JUST BEING ABLE -- I DON'T KNOW, YOU MIGHT BE ABLE TO SAY A FEW WORDS ABOUT WHAT IT TOOK FOR YOU THINK A LOT OF TIMES IN OUR BIGGER AGENCIES THERE IS A WHOLE -- YOU KNOW, IT'S DIFFICULT TO MAKE A CHANGE FROM THE LEFT TO THE RIGHT. LIKE WHAT KIND OF CHALLENGES DID YOU HAVE TO FACE TO DO THAT? GETTING THE TRAINING IN THE DATA MANAGEMENT, COMMERCE DATA ECONOMY WAS INSTRUMENTAL IN THAT FOCUS ON WRITING THE CODE TO DO IT, AND MY LEADERSHIP WAS IN SUPPORT OF IT. THEY SAW KIND OF WHERE THE COMMERCE DATA ECONOMY WAS LOOKING TO GO WITH IT, AND SO, YOU KNOW, THAT WAS A MAJOR ENABLEER, AND THEN JUST LEARNING ACQUIRING FEEDBACK, BUILDING OUT ITERATIVELY HELPED MAKE IT MORE MANAGEABLE SO IT WASN'T JUST CLIMBING THE MOUNTAIN RIGHT OUT OF THE GATE. AND THEN I'LL SAY IT AGAIN -- INSTALLED. SO WE HAD A LONG FIGHT WITH OCIO AND IT TOOK A LONG TIME, BUT THEY CAME IN AND KIND OF HELPED EDUCATE FOLKS THAT THESE WERE USEFUL TOOLS AND THAT GOT US THROUGH THE GATE. SPEAKER: A LITTLE EDITORIAL ON THAT. I WOULDN'T SAY THE OFFICE OF THE CHIEF INFORMATION OFFICER THAT WE FOUGHT WITH THEM AS MUCH AS PARTNERED WITH THEM AND COLLABORATED WITH THEM. (LAUGHTER) . SPEAKER: STEVE COOPER PRESENTED THIRD CDAC MEETING, HIS VISION WITH EMPOWERING THE DEPARTMENT WITH INFRASTRUCTURE HAS BEEN A GREAT SUPPORT, BUT, YEAH, R WAS JOWLT LOWED, GITHUB WAS OUTLAWED OUTLAWED, APPROXIMATE SOCRATA WASN'T ALLOWED, NOT ALLOWED TO BE USED, FIREWALLED, LET'S SAY TECHNICALLY FIREWALLED, THAT AS A LARGE PART CHANGED. BUT JUST GIVING A SALE WITHIN GOVERNMENT, GIVE US R, OBVIOUSLY FAILED. SAYING STEVEN TR -- DISEMPOWERED AGENCIES RKTS CAN SECURITY ASSESSMENTS ALREADY PERFORMED, THAT IS A MUCH EASIER CONVERSATION AND VERY GRATEFUL, I DON'T KNOW IF ANY FOLKS FROM S SAO ARE HERE, BUT ALL THESE TOOLS ARE FIRST-TIME IN THE DEPARTMENT IN JUST THE PAST YEAR YEAR. SPEAKER: ALWAYS AMAZING TO SEE A TOOL WHICH LOOKS LIKE BAD REPLACED BY PRETTY MUCH ANYTHING ELSE, BUT CERTAINLY -- (LAUGHTER). SO THE QUICK QUESTION I HAD WAS IT WOULD BE GREAT, AND I THINK IT'S WORTHWHILE FOR ALL THE PROJECTS YOU ACTUALLY UNDERTAKE THROUGHOUT THE DEPARTMENT IS TO TALK ABOUT IF IT'S POSSIBLE TO MEASURE THE IMPACT ON THE RECIPIENTS OF THE GRANTS OR WHATEVER YOUR SPECIFIC GOALS ARE THESE KINDS OF TOOLS ARE SUPER- SUPER-VALUABLE, BUT IF YOU'VE GOT AN IDEA, SPECIFIC IDEAS OF LIKE WHAT NUMBERS YOU'RE TRYING TO MOVE IN TERMS OF GRANTS CLOSING OR THE TIME TO CLOSE A GLANLT, AGAIN, I DON'T KNOW WHAT YOUR KPI'S ACTUALLY ARE, BUT TO PROBABLY ENCOURAGE, SAY, THE RIGHT DEFAULT GRAPHS OR THE RIGHT SORT OF READILY ACCESSIBLE ACTUALLY DOING THE WORK, BECAUSE THIS STILL REQUIRES, I BET, A CERTAIN AMOUNT OF, YOU KNOW, SELF-DIRECTION AND YOU HAVE TO KNOW HOW TO USE THE TOOL AND SO FORTH. MY BIG POINT IS IF YOU UNDERSTAND LIKE WHAT YOU'RE TRYING TO DRIVE FOR THE RECIPIENTS OF THE GRANTS, THEN YOU CAN BUILD THE BACK INTO THE DESIGN OF THE APPLICATION IN A WAY WHICH MAKES IT EVEN MORE VALUABLE FOR THE ACTUAL BUSINESS GREAT POINT. SPEAKER: I KNOW WE'RE SHORT ON TIME, I'LL JUST THROW ONE MORE TOPPER ON THIS. STEVEN IS ALWAYS VERY MOD ESES MODEST IN THIS PRESENTATION, BUT THE IDEA THAT WE MOVE FROM THE LEFT TO THE RIGHT IN BASICALLY THREE OR FOUR MONTHS AND IT WAS DRIVEN DIRECTLY BY THE BUREAU TO SOLVE A PROBLEM, THE REACTION FROM THE LEADERSHIP OF THE BUREAU AND FROM THE TEAM HAS BEEN TREMENDOUS BECAUSE LOOK, THE ABILITY TO ACTUALLY UFS THE DATA BE ABLE TO RESPOND QUICKLY WHEN THEY NEED TOLL HAS BEEN REALLY WELL RECEIVED, YOU HAVE TO HAVE GREAT LEADERS LIKE JAY WILLIAMS WHO PUT THE ARE EFFORT INTO THE APPRENTICESHIP PROGRAM AND HAVE IT DRIVEN OUT AND HAVE STEVEN BE SO AWARE AND BE ABLE TO DRIVE CHANG ALL OVER THE AGENCY HAS BEEN TREMENDOUS. STEPHEN, THANK YOU VERY MUCH, AND OBVIOUSLY THE CDS TEAM FOR BEING YOUR PARTNER. SPEAKER CLORN THANK YOU SPEAKER: I WAS SUPPOSED TO START BY TELLING UP THE #FOR THE MEETING IS CDACMEETING, IT'S AT THE BOTTOM OF IT. IT'S PRETTY FUN TO BE AT A MEETING WHERE WE'RE TALKING ABOUT R, SHINY, PYTHON, AND HAVING OUR TEAM MEMBERS AROUND HERE TALK ABOUT AGILE DEVELOPMENT, ALPHA AND BETA TESTING. IT'S A NICE PLACE THOSE WHO ARE FOLLOWING US ONLINE, THAT HASHTAG IS IT LIVE, WHVMENT WE GET TO THE PUBLIC COMMENT PERIOD, IF YOU HAVE QUESTIONS, IF YOU HAVE COMMENTS, WE'LL PULL FROM THAT TO HELP INTERJECT THEM INTEUFER THE DISCUSSION. SO THAT PUBLIC COMMENT ISN'T JUST OPEN TO PEOPLE IN THE ROOM, IT'S OPEN TO THE PEOPLE ONLINE THROUGH TWITTER. SPEAKER: HERE WE HAVE WILL TALK THROUGH CLUSTER MAPS OF PATENT APPLICATIONS. KARL? WILL KARL: GOOD MORNING, MY NAME IS CACIALTION I'M A SUPERVISOR AT THE PATENT AND TRADEMARK OFFICE. I'LL TALK A LITTLE BIT ABOUTLY ABOUTLY OUR PATENT DATA THAT WE HAVE GOT, ROUGHLY EVERY YEAR THE OFFICE RECEIVES ABOUT HALF A MILLION PATENT APPLICATIONS , AND EACH PATENT APPLICATION GETS A CLASSIFICATION SO WE KNOW WHERE IT SHOULD GO, THE APPLICATIONS ARE PUT ON A LIST AND SENT TO AN EXAMINER SUPERVISOR LIKE ME. I THEN MANUALLY ASSIGN THOSE CASES CASES. WE'VE BEEN DOING THAT BY USPC AND ONE PROBLEM IS THAT WE'VE STOPPED USING USPC FOR CLASSIFICATION, WE'VE CHANGED OVER TO CPC, WHICH IS NOW STANDING FOR THE COOPERATIVE PATENT CLASSIFICATION SYSTEM, SHARED BY THE MAJOR PATENT OFFICES IN THE WORLD. BUT THIS PROCESS WHERE I HAVE TO MANUALLY ASSIGN IS SLOW AND INACCURATE AND INEFFICIENT. SO TO TALK ABOUT THE EXPLORATION IS WHERE I'M GOING TO GO NEXT, SO TO START THIS, I'M SHOWING YOU A MILLION PATENTS ISSUED IN THE LAST THREE YEARS BY THEIR CPC CLASSIFICATION AND TO WHICH WORKING GROUP ISSUED THEM. AND WHAT I WANT YOU TO TAKE HOME FROM IT IS MASSIVELY INTER INTERCONNECTED. SO HOW DO WE MAKE SENSE OF THAT? I'M GOING ON DRILL IN CLOSER TO THE CLASSIFICATION FOR SEMICONDUCTOR MANUFACTURER, AND EVEN IN THAT SPREAD ACROSS THE OFFICE, THE MAP HAS DIFFERENT AREAS OF THE OFFICE DOWN IN THE LOWER RIGHT-HAND CORNER ARE THE BIOLOGICAL, BIOTECH AREAS. UP AT THE TOP ARE THE ELECTRICAL ENGINEERING, AND ON THE RIGHT -- DRILLING INTO IT. OH, I WENT TOO FAR. OKAY. THERE'S A SLIDE HIDDEN WHICH SHOWS THE INTER INTERCONNECTIONS BETWEEN THE PATENTS THAT SITE THIS CLASSIFICATION AND THE OTHER CLASSIFICATIONS THAT ARE ISSUED ONTO THAT PATENT. AND WHAT THE BIG TAKE-HOME IS THAT EVERY PATENT OR APPLICATION THAT COMES CLASSIFICATIONS THAT WE CAN USE TO IDENTIFY WHAT SUBJECT IS DISCUSSED IN THAT APPLICATION. SO THE IDEA CAME UP, CAN WE LOOK AS A PROXY FOR OUR EXAMINERS' EXPERIENCE? AND THE WAY THIS WORKS IS WE TAKE ALL THE APPLICATIONS THAT HAVE BEEN EXAMINEED, GENERATE A FINGERPRINT FROM FOR EVERY EXAMINER COMPL, AND THEN LOOK AT INCOMING APPLICATIONS WITH THE CPC CLASSIFICATION AND MATCH THEM UP WHICH EXAMINER WOULD BE THE MOST APPROPRIATE ONE FOR THAT INCOMING APPLICATION. SO THIS SLIDE HERE SHOWS THE CONTRAST BETWEEN WHAT WE'RE CURRENTLY DOING WITH AN EXCEL SHEET COMING THIS WHICH I HAVE TO GO THROUGH MANUALLY AND SAY OKAY, THIS EXAMINER EXAMINER, YOU'RE DOING ALLLY APPLICATIONS THAT END IN ZERO, HERE YOU GO, YOU DO THE ONES THAT END IN ONE, AND GO THROUGH. SO I'M NOT REALLY USEING -- WHAT I'VE GOT BEHIND IN MY GROUP. ON THE RIGHT IS A MAP THAT SHOWS SOMEONE'S EXPERTISE, HOW IT'S LINKED BETWEEN DIFFERENT KLATT FIXES. SO IF I USE THAT, I CAN HONE IN ON WHO SHOULD BE -- HONE IN ON THAT RECOMMENDATION. SO THE END RESULT IS WE COULD AUTOMATE THIS USING THE KNOWLEDGE MANAGEMENT AND THAT'S GOING TO HELP US DRIVE QUALITY OF PRODUCTS, THE ACTUAL PATENTS UP BECAUSE WE'RE GOING TO FIND THE MOST APPROPRIATE EXAMINER AND HOPEFULLY THEY'LL BE MORE EFFECTIVE. THANK YOU. SPEAKER: ALL THAT'S TERRIFIC. C. J. AND COLIN, YOU'RE LEADING THIS ONE. C. J.: FIRST AS A PATENT HOLDER AND REOCCURRING APPLICANT, I WOULD LIKE TO THANK YOU, CARM, FOR YOUR WORK IN THIS SPACE. I COULDN'T HELP BUT NOTICE BETWEEN ONE OF THE PREVIOUS DISCUSSIONS THE SEARCH ANALYSIS THAT JEFF AND TEAM WERE WALKING ON THAT THE CLASSIFICATION CONNECTIONS THAT YOU HAVE COULD ACTUALLY GO BACK TO SUPPORTING THEIR CATEGORIZATION, SO I THINK THERE'S SYNERGY THAT COULD BE HAD BASED UPON THE ANALYSIS THAT TO MAKING SURE YOU HAVE GOOD CATEGORIZATIONS TO BE ABLE TO CROSS-SEARCH THEM. SO THAT'S ONE THING THAT WORKING TOGETHER WITH THAT TEAM MAY PROVIDE SOME VALUE. AND THEN JUST ONE QUESTION, I COULDN'T HELP BUT HEAR THAT IT'S A LOT OF THE TERMS COULD BE, IF WE WERE TO, WHAT'S THE POSSIBILITIES OF SEEING THIS IMPLEMENTED, AND I DON'T KNOW IF IT'S A QUESTION FOR YOU OR FOR TOM, BUT ESSENTIALLY -- WHAT DO YOU NEED TO MOVE THAT FORWARD? SPEAKER: WE'RE CURRENTLY WORKING ON HOW TO IMPLEMENT THIS. C. J.: IT LOOKED LIKE AN EXCELLENT PROOF OF CONCEPT. KARL: THAT'S WHAT IT IS. C. J.: THAT'S GREAT. I THINK IT'S ONE OF THE THINGS THAT GETTING THOSE PROOF OF CONCEPTS, A CHASM BETWEEN A PROOF CONCEPT TO ACTUALLY GETTING IT IMPLY MENLTD, I THINK THAT GOES BACK TO NUMBER ONE RECOMMENDATION OF INSTITUTIONAL INSTITUTIONALIZATION AND THAT IS MAKING SURE WE MAKE THAT JUMP FROM THERE AND POTENTIALLY TYING STRING ANALYSIS AND BUNLD LING SOME OF THESE THINGS TOGETHER SO OF TOOLS THAT THEN BECOME THAT CAPABILITY TO ACTUALLY STREAMLINE THE ENTIRE PROCESS FROM BEGINNING TO END AND CREATE PUT IT TOGETHER, I HAD MY EYE ON IMPLEMENTATION, SO EVERYTHING THAT I'VE WRITTEN I THINK CAN BE FOLKS. SPEAKER: I FULLY AGREE WITH C. J. I MEAN, WHAT WOULD POSSIBLY STOP YOU GOING FROM ZERO TO ONE. THAT'S GREAT. I ALSO LOOK AT IT AND I SEE SOMETHING A LOT LARGER HERE. WHAT I UNDERSTAND YOU SAID IS THAT WHAT WE DO IS WE CHARACTERIZE, WE'VE GOT A FINGERPRINT OF THE PATENT AND A FINGERPRINT OF THE EXAMINER AND MATCH IT. SO IF I HAVE A FINGERPRINT OF THE PATENT, I ALSO HAVE A FINGERPRINT OF MAYBE MATCH THAT TOO OR I CAN HAVE A FINGERPRINT OF THE SKILLS I NEED CAPABILITY YOU HAVE AND WHAT YOU'VE DONE IS CHARACTERIZED THIS ON A PLATFORM TO MATCH NEARLY ANYTHING THAT SOMEONE COULD THINK ABOUT. SO THAT'S WHY I FULLY AGREE WITH C. J., I MEAN, LET'S DO T LET'S PUT IT OUT THERE, AND EVERYBODY WILL -- VARIOUS PEOPLE, THESE SAME THEMES COME UP. OH, YOU COULD USE IT THIS WAY. YEAH, YOU COULD USE IT THAT WAY. YOU COULD USE IT TO DRIVE HIRING. YOU COULD USE IT TO DECIDE WHERE APPLICATION, IT IS A WIDE- WIDE-RANGING PROJECT. COLIN: GREAT JOB. KARL: THANK YOU. SPEAKER: LET ME JUST SAY TO THE TERMS OF THE FIRST ROUND, EVEN THE FIRST ROUND OF FOLKS WHO WERE DETAILED INTO THE CDS TO WORK ON PROJECTS, WE COULD DO AN HOUR-LONG PRESENTATION IN 8 OF THE 15. I WILL TELL YOU THEY DIDN'T ALL WORK OUT PERFECTLY, AND WE ARE OKAY WITH THAT. WE'RE NOT GOING TO SHOW YOU EVERY SINGLE ONE AND GO THROUGH THEM, BUT I'M TELLING YOU NOT ALL OF THEM WORK PERFECTLY, WE'RE SHOWING YOU SOME OF THE ONES THAT WERE REALLY SUCCESSFUL SUCCESSFUL, BUT A LOT OF THEM WERE SUCCESSFUL, AND WHAT THE TEAM IS DOING TO TRY AND SCALE, RIGHT, TO INSTITUTIONALIZE AND SCALE IS WE HAVE A GROUP OF SOME OF OUR -- AND KARL IS ONE OF THEM, STEPHEN IS ONE OF THEM, THERE ARE OTHERS, WHO WE ARE DOING ROAD SHOWS, WE GO AROUND TO THE DEPARTMENTS, TO THE OCIO COUNCIL, TO THE LEADERSHIP COUNCIL SAYS, TO THE DEPARTMENT DEPARTMENTAL LEAD COUNCILS INTO THE BUREAUS AND WE SHOW THEM THIS IS WHAT WE CAN DO FOR YOU, RIGHT? THESE ARE THE DIFFERENT PRODUCTS, THESE ARE SCALEABLE, THESE ARE MORE CUSTOMIZED, BUT THAT'S WHAT'S DRIVEN THE WORKLOAD THAT WE HAVE NOW SCOPED OUT FOR THE NEXT YEAR, AND WE'LL TALK ABOUT THIS LATER, BUT USUAL YOU'LL SEE, WE HAVE A LOT OF WORK KIND OF COMMITTED FOR NEXT YEAR. WE HAVE TO BUILD CAPACITY AND EXECUTE. THEALTSES OUR BIG CHALLENGE. BUT WE'RE PROUD OF THIS, AND WE'RE TRYING TO SHOW IT EVERYWHERE WE CAN, AND WE NEED YOUR HELP TO AMPLIFY REALLY REALLY, YOU KNOW. SO MAYBE I CAN GO TWO MORE. SPEAKER: THIS IS FANTASTIC AND I CAN'T HELP BUT DRAW THE CORRELATION DIRECTLY BETWEEN I WORK IN SCIENTIFIC RESEARCH AND HAVE BEEN A JOURNAL EDITOR AND PUBLISHER, AND THE ONE TO ONE MATCH BETWEEN PATENT EXAMINER AND THE PATENT, IT LOOKS LIKE IT PEER REVIEWER AND JOURNAL, I WONDER IF YOU'VE TALKED TO THE SCIENTIFIC COMMUNITY, THE THORNY IDEA IS THE TIME TO GET IDEAS TO MARKET, THE TIME IT TAKES TO MATCH UP GOOD MATCHES IN PEER REVIEW SPEAKER: I HAVE NOT. SPEAKER: WE WOULD LOVE TO TALK TO YOU. SPEAKER: I KNOW THERE ARE SOME COLLEAGUES AT AN OFFICE GRANT REVIEWERS WITH INCOMING GRANTS, SO -- SPEAKER: HEAD AND DEALING WITH NOW. MIGHT WANT TO REACH OUT, OR WE'LL REACH OUT TO YOU. CL SPEAKER: BASICALLY THE SAME COMMENT. THE ASSIGNMENT OF TO WHATEVER YOU CARE ABOUT, BE IT PAPERS OR GRANT PROPOSALS, HAS BEEN DONE, AND I THINK YOU WOULD MIGHT LIKE TO LOOK AT SOME HAVE DESIGNED INTERFACE THAT IS ARE EASY TO USE, HOUR THEY GO ABOUT UPDATEING FINGERPRINTS, HOW THEY GO ABOUT USING THEM, THE MORE ADVANCED MECHANISMS TO DO THE LINGUISTIC ANALYSIS, SO I'M HAPPY TO CONNECT YOU THERE. SPEAKER: THANK YOU. SPEAKER: THAT WAS TERRIFIC. WHY DON'T WE BEACH FROM PTO. TOM BEACH: THANK YOU, JUSTIN. I WANT TO START OUT AND THANK YOU CDAC AND EVERYONE HERE FOR THE OPPORTUNITY. I FEEL LIKE IT'S ALMOST BEEN PREPLANLD HERE -- PREPLANNED HERE TO TEE THIS ONE UP. FOR THOSE WHO KNOW, I'VE SPOKEN FOR CDAC A COUPLE TIMES, JUST WHAT'S HAPPENING TODAY REPRESENTS THE CONSULT CULTIVATION GOING ON AT THE USPTO, TO REMEMBER WAY A ROAD MAP, AND THAT WAS A SMALL SMALL, SMALL TEAM, AND I WOULD BE REMISS TO SAY THAT THERE'S A LARGE TEAM NOW BEHIND ALL THESE DIFFERENT EFFORTS. AND I THINK IT'S A GREAT, GREAT THING. ONE OF THE THINGS I WAS TELLING IAN IS THAT WE JUST HIRED OUR FIRST DATA SCIENTIST FULL-TIME GOVERNMENT, WE'RE LOOKING TO HIRE OUR SECRETARY, SO WE ARE TRULY ON A JOURNEY, AND I WOULD LIKE TO THANK YOU ALL FOR THAT OPPORTUNITY. SO WITH THAT KILLED LIKE TO MOVE INTO TWO TOPICS, ONE YOU MAY OR MAY NOT HAVE HEARD OF, THE USPTO CANCER MOONSHOT, HOW DID WE PLAY A ROLE MAY NOT THINK THE USPTO IS INNATELY, THIS IS REALLY SENSITIVE, INNATELY PART OF THE CANCER MOONSHOT PROGRAM BECAUSE WE'RE TALKING ABOUT A LOT OF DATA THAT'S DRIVEN ON CANCER THINGS THAT WE'VE LOOKED AT, USP USPTO, AS A LEADER INDICATOR OF TECHNOLOGY BASED ON OUR PATENT FILINGS, SO WHAT YOU SEE BEING FILED IS AN INDICATOR OF WHAT IS GOING TO COME TO MARKET. SO I WAS PART OF THE TASK FORCE TEAM ON DATA, AND WE LOOKED AT AN EFFORT TO LEVERAGE TWO THINGS WE'VE NEVER DONE BEFORE, ONE WAS A CHALLENGE.GOV EFFORT, WE HAVE GRANT AUTHORITY AT THE USPTO, AS CHIEF ECONOMIST OFFICE, OUR CHIEF COMMUNICATIONS OFFICE, AS WELL AS OUR GC GENERAL COUNCIL, TO ALLOW US TO SORT OF STAY WITHIN THE PATH OF A VISION, AND OUR VISION WAS TO TAKE WHAT YOU'VE SEEN BEFORE, OUR DEVELOPER HUB, USE THAT PLATFORM PLATFORM, AND PLATFORM IS A THEME FOR US. I'LL TOUCH ON THAT THE IN NEXT TOPIC TOORK AS A PLATFORM, THAT CONCEPT WHERE YOU CAN TAKE T USE IT, IT'S PORTABLE, AND IT CAN BE LEVERAGED IN MANY UNIQUE WAYS. THIS WAS A GREAT OPPORTUNITY FOR US OF BEING ABLE TO SAY WE'RE DOING PLATFORM WORK AND NOT IN A TECHNICAL THINGS GOING ON BEHIND THE SCENES, BUT AS A PLATFORM, WE'RE ABLE TO DO SOMETHING LIKE THIS WHERE IN ABOUT FOUR WEEKS WE PUT OUT A CHALLENGE.GOV STATEMENT, WE GOT 19 SUBMISSIONS IN FOUR WEEKS FOR A VERY NOMINAL PRIZE MONEY, I MIGHT ADD, SO WHAT DID IT LOOK LIKE? OKAY. SO THE RESULT SETS THAT WE GOT. WE WERE CHALLENGING FOLKS TO SAY HOW WOULD YOU ACT IF YOU WERE THE HEAD OF A GOVERNMENT AGENCY AND HAD TO MAKE A DATA-DRIVEN POLICY DECISION ABOUT WHERE TO SPEND MONEY, RIGHT? LIMITED RESOURCES. WHERE WOULD YOU SPEND MONEY IN THE WORLD OF CANCER TREATMENT? SO THAT WAS THE CHALLENGE, AND IT WAS REALLY INTERESTING TO SEE HOW THEY COULD TAKE OUR CURATEED DATA, THE IMPORTANCE ON CUREATION, WE USE CLASSIFICATION SYSTEM, BUT ON TOP OF THAT WE HAVE INSTITUTION INSTITUTIONAL KNOWLEDGE, WHICH WAS DEMONSTRATEED BY THE SOURCE, A HYBRID. SO FOR THE FIRST TIME WE DID A HYBRID VERSION OF A CURATEED DATASET, CONVERTED IT ONTO AN API, THREW IT UP ON DEVELOPER HUB, SET IT LOOSE, AND REALLY FASCINATING WAYS OF TAKING CLINICAL TRIAL DATA, MORTALITY RATES, NIH FUNDING, PREDICTIONS OF WHERE TECHNOLOGY IS GOING, USING SOMETHING CALLED THE ORANGE BOOK WHICH TELLS YOU WHERE A TRACE STUDY OF COMMERCIALIZATION FROM PATENT TO PRODUCT, IT SEES IT IN THE PRESCRIPTIONS, UNDER MEDICATIONS SHORT TIME FRAME, THEY WERE ABLE TO ACTUALLY BUILD FULLY FUNCTIONING MODELS THAT CAN BE USED TO MAKE THESE DECISIONS. YOU KNOW, SHOULD WE FUND AT MEDIUM LEVELS -- MEDIAN LEVELS THAT WERE ESTABLISHED ABOVE OR BELOW FOR AN IMPACT AT THIS HIGH LEVEL OR THIS LEVEL TO A GIVEN AUDIENCE THAT IS DEALING WITH CANCER? SO THERE'S A REALLY FASCINATING ATTEMPT TO REALLY PUT IT OUT THERE, HAVE THEM CROWD AND SOURCE OUR DATA, WHAT WE HAVE NORMALLY THOUGHT. ONE OF THE THINGS IN THE OPENING REMARKS FROM SECRETARY PRITZKER ABOUT CREATING NEW IDEAS AND CREATING A CULTURAL SORT OF REVOLUTION, WHAT WE'RE TRYING TO WORK THAT WE DO BUT ALSO NOT JUST BE DISRUPTIVE FOR A SNG HE WILL STORM, WE WANT IT TO BE CHANGE AND CHANGE THAT'S EMBEDDED. SO NEXT STEPS, WE'RE CONTINUING ON WORKING WITH THOSE PROVIDED SOME OF THE WINNING TEAMS, SOME OF THE OTHER TEAMS. SOMETHING CALLED A DATA API TOASTER. I'M KNOWN WITH COMING UP WITH VERY STRANGE TERMS, SO YOU'LL HAVE TO FORGIVE ME. BASICALLY WHEN WE LOOKED AT HAVING A CURATEED DATASET, SOIN STEAD OF JUST OPENING DATA AND YET WE REALLY WANT CONSTRUCTIVE USEFUL DATA, SOMETIMES FOLKS COME IN EITHER WHETHER IT'S WITHIN THE INSTITUTION OR THEY COME WITH A KNOWN LEVEL OF EXPERT TITION OF RCHING -- EXPERTISE OF KNOWING WHAT THEY WANT. SO WHAT MADE THEM COME, CURATE THROUGH OUR SEARCH AND TOAST IT INTO AN API AND GO BACK HOME WITH IT. THAT'S SOMETHING WE'RE LOOK BEING AT WORKING ON AS A NEW DELIVERABLE IN THE OPEN DATA SPACE IS JUST SORT OF THIS SELF-SERVE API DEVELOPMENT. IT GOES TO THAT EFFECT OF LOOKING AT NOT ONLY JUST CREATING AND BEING PRESCRIPTIVE ON API'S BUT ALSO LETTING THEM COME AND CREATE THEIR OWN. SO WHY DO WE DO ALL THIS? THE BIG THEME FOR US ON OPENING DATA IS ALLOWING US TO HAVE BETTER APPLICANTS, AND BETTER APPLICANTS ARE FOLKS THAT BETTER UNDERSTAND THE PATENT SYSTEM AND IT'S A FEEDBACK GROUP SYSTEM AND I THINK WE'RE SEEING IT IN REALTIME AS MORE AND MORE FOLKS UNDERSTAND WHAT PATENTS ARE, WHAT ROLE THEY PLAY, WE'RE GOING APPLICATIONS. WITH THAT, I'LL PIVOT ON SOMETHING WHAT'S COMING COMING, WHAT'S NEW AND EXCITING THAT I'M REALLY EXCITED ABOUT, THIS SIDE OF THE HOUSE WE'RE LOOKING AT A PLATFORM CALLED SIGMA PLUS PLUS AND A COUPLE QUESTIONS CAME UP EARLIER, HOW CAN WE INTEGRATE DIFFERENT THINGS INTO OUR SEARCH SYSTEM. WE'RE LOOKING AT A BIG DATA ANALYTIC SEARCH SYSTEM AND I'M GET TO UI JUST TO START BACKWARDS. REMEMBER THAT THIS IS A BIG DATA APPROACH WHERE KNOWLEDGE MANAGEMENT IS KEY, WE'RE LOOKING AT A UI THAT LOOKS BASICALLY LIKE THIS, LEVERAGING A I ALGORITHMS, WORD NET, ANY SORT OF STEMMING, LEMMING PROCESSES, THIS IS USABLE BY THE PATENTS BUT LIKE A CARD DECK, YOU'RE SHUFFLING THE DECK TO HAVE THE MOST RELEVANT ART AT THE TOP. IT CONTINUOUSLY RE RECYCLES AND RESHUFFLES. THE WORD CLOUD IS A SORT OF QUALITY CHECK. IF YOU SEE THE IMPORTANT WORDS OF THE THINGS APERTAIN TO THE THINGS YOU'RE LOOKING FOR, YOU'LL SEE THEM IN DENSITY IN COLOR. LOWER LEFT, YOU'LL SEE SOMETHING CALLED BOOSTERS WHERE THEY CAN ACTUALLY CHANGE IN VALUE OR REDUCE OR INCREASE THE VALUE OF SECTIONS, SO CLAIM LANGUAGE AND PARTS OF THE APPLICATION. WE'RE ABLE TO TAKE -- SP GET AWAY FROM JUST TYPE SEARCHING AND DO SHINGLES AND GRAPHS AND PULL APART THE MULTIPLE DIMENSIONS . WE'RE LOOK AT WHAT WILL THE PATENT EXAMINER TO USE? SOMETHING LIKE THIS IS WHAT WE'RE LOOKING AT, WE'RE LOOKING AT ITS CAPABILITY OF PROVIDING FEEDBACK, SO AS IT RE RESHUFFLELESS THE DECK, THE EXAMINER CAN SAY I LIKE THIS, I LIKE, I DON'T LIKE, CONTINUALLY ADDING TO THE SYSTEM, THAT'S THE MASS TREE ISSUE EXRRL. AS WELL AS CLASSIFICATION AND THE WORK THAT KARL IS DOING, IT WILL EVENTUALLY COME TOGETHER UNDER THIS ENTIRE PLATFORM. THAT'S THE VISION AND I IMAGINE I'M RUN RUNNING OUT OF TIME, BUT I DO WANT TO SAY THAT, YOU KNOW, WITHOUT CDS AND WITHOUT YOUR ALL CDAC'S SUPPORT AND INTEREST, WE LITERALLY STARTED TWO AND A HALF WHERE WE ARE TODAY. SO IT'S A MAGNIFICENT RIDE AND I CAN'T THANK EVERYBODY ENOUGH TO ALLOW US TO HAVE THE OPPORTUNITY TO BE SPEAKER: THAT'S TREMENDOUS. I'LL TURN IT OVER TO KEVIN AND VADIM. SPEAKER: A COMPLIMENT AND QUESTION P THE COMPLIMENT IS WEIGHT CLASS. I REMEMBER THE PRESENTATION WHERE YOU 140ED THE YOU ARE EXECUTEING AGAINST THE ROAD MAP QUITE WELL. IN ONE OF THE PRESENTATIONS EARLIER TODAY, I THINK THE STATISTICS THAT JEFF SHARED ARE SOMEWHERE BETWEEN HALF A MILLION AND 600,000 APPLICATIONS PER YEAR, FIVE HOURS PER APPLICATION, 8,000 PATENT EXAMINERS, DO YOU THINK SIGMA PLUS PLUS HAS THE OPPORTUNITY TO EITHER REDUCE THE AMOUNT OF TIME THAT IT TAKES EACH PATENT EXAMINER TO EXAMINE AN APPLICATION AND THEREFORE THE KIND OF RELATED QUESTION IS CAN WE AS INDUSTRY EXPECT PATENTS TO APPLICATION PROCESS FASTER? AND THEN THE LAST COMMENT THAT I WOULD MAKE FOR YOU IS IF YOU REMEMBER KARL'S PRESENTATION WHERE WE LOOKED AT THE TABBED OLD UI, THAT WAS PROBABLY ABOUT TEN YEARS OLD, SO I'M A LITTLE CONCERNED THAT YOU'RE THINKING NOW OF THE SIGMA PLUS PLUS AS BEING SOMETHING THAT'S GOING TO BE A DECADE FROM NOW, BECAUSE A DECADE FROM NOW, THE UI IS PROBABLY GOING TO BE AN A I- I-BASED UI THAT COMES FROM ALEXA OR SOMETHING, SO I WOULD BE THINKING ABOUT MAYBE THAT CAN BE LOOK AT THIS AS A COGNITIVE ASSISTANCE, THAT'S THE PARADIGM FOR WHAT WE'RE LOOKING AT THIS. THERE'S A LOT OF TALK IN THE INDUSTRY OF THE CHAUFFEUR VERSUS RIGHT NOW. THESE TYPES OF TOOLS ARE CO-PILOTING EXAMINER TO MAKE THE HIGHEST QUALITY ASSERTION OF PATENTABILITY, SO THAT ISN'T REALLY RELATIVE TO TIME OF THE THAT'S ENABLING A MORE EFFECTIVE DECISION TO BE MADE. SO OUR GOAL IS NOT TO SIT HERE AND GO HOW MUCH FASTER CAN WE GO. OUR GOAL IS TO RAISE THE BAR OF QUALITY. SO IN ORDER TO DO THAT THAT, WE'RE LOOKING AT COGNITIVE THERE ISN'T GOING TO BE A REPLACEMENT FACTOR. THIS ISN'T WORK TO MOVE AWAY FROM THE HUMAN ELEMENT OF BEING ABLE TO DECIDE PATENTABLE SUBJECT MATTER. SO I ISSUE. IT'S NOT OUR MISSION. BUT ON THE UI FRONT, YES, THIS IS RATHER, YOU KNOW, WELL, ACTUALLY, IT'S WITHIN OUR STANDARD CODE STANDARDS FOR THE OFFICE TO USE THIS SORT OF LOOK AND FEEL. NOW, WHAT WAS DRIVING ON THIS IS THAT IT'S OPEN SOURCE PLATFORM MODULAR AND WILL CHANGE OVER TIME, AND WHEN I MENTION THINGS LIKE PLATFORM AND FUNCTIONALITY, WE'RE REALLY TRYING TO LOOK AT DEVELOPING THINGS THAT WILL MATTER TO THE EXAMINER TEN YEARS FROM NOW AND HOW THEY'RE EXPRESSED IN THE UI, THAT CERTAINLY CAN CHANGE OVER TIME. ABSOLUTELY. BUT WE WANT THEM TO BE MORE INTERACTIVE WITH THEIR WORK, AND WE ALSO WANT TO EXTRACT THAT INFORMATION ON THE BACK END. SO EVERY TIME THEY USE IT, WE FINGERPRINT T PULL IT AND GOES, WELL, THIS OTHER EXAMINER USED TO DO THIS WHEN THEY DID IT. THAT'S THE LOOP. HI HOPE THAT HELPS. SPEAKER: SIGMA PLUS PLUS UI, I THINK THE DATA WHICH IS BEHIND IT IS VERY, GOES BACK HUNDREDS OF YEARS, BUT AFTER ALL, WE JUST GOT STARTED. WHEN YOU THINK ABOUT THAT, HOW MUCH WENT INTO TRANSFORMING HOW DATA IS STOREED BEHIND THE SCENE , IT REALLY BRINGS MAGNITUDE OF THE WORK WHICH WAS DONE HERE. THIS IS JUST A CULMINATION OF A LOT OF WORK WHICH WAS DONE. IT'S I THINK GREAT UI, THERE ARE SOME OPTIONS HERE WHICH WHEN I HEARD ABOUT T I WAS LIKE, WOW, SO WHEN YOU HEAR OF PATENTS, ONE PATENT RELATES TO ANOTHER ONE, YOU CAN LOOK AT THE PATTERN OF IN PATENTS, LIKE YOU DO E-MAIL IN YOUR THREADS, IT'S ALSO HERE. WOULD BE GREAT, IT WOULD BE GREAT FOR THE PUBLIC WHEN WE SEARCH FOR OUR PATENTS, NOT TO HAVE TO GO THROUGH 50 PATENTS AND FIGURE OUT WHICH ONE IS WHICH. BUT THE AMOUNT OF WORK WHICH WENT BEHIND, IT'S PROBABLY SUMMARIZED IN THE PREVIOUS SLIDE SLIDE, SOME OF THE TECHNOLOGY IS JUST STARTING THE WORK AS A ROAD MAP, AND ACTUALLY FROM TWO YEARS AGO, I CAN SAY FOR THE LAST SET OF COMPANIES IT ALSO WOULD BE WOW AND I THINK IT'S A GREAT WORK CHOFTION DONE AND IF YOU CAN -- GREAT WORK WHICH WAS DONE AND IF YOU CAN MAKE SHOWCASE OUT OF IT AND SHOW OTHER DEPARTMENTS AND COMPANIES WHAT CAN BE DONE IN TWO YEARS AND THE RESULT AND THE RESULT HERE, I THINK IT WILL BE GREAT. SPEAKER: AGREED. THANK YOU. WE WOULD LOVE TO DO THAT MORE. ABSOLUTELY. SPEAKER SPEAKER: A MINUTE OR SO. DOES ANYBODY ELSE WANT TO ASK ANY QUESTIONS ON THIS ONE? GO AHEAD AHEAD. SPEAKER: FOLLOWING UP ON KEVIN'S QUESTION ABOUT I GUESS I WAS ASSUMING THIS WAS ABOUT EFFICIENCY, BUT I THINK YOU SAID IT WAS ABOUT QUALITY,, NOT EFFICIENCY. WHAT IS QUALITY WOULD NOT ISSUE A PATENT THAT ACTUALLY IN FACT HAD PRIOR ART? THAT? SPEAKER: YEAH. SO WE OFFICIALLY DEFINE QUALITY AS OUR IT'S DEFINED AS USING ART FOR PATENT EXAMINATION PROCEDURE. THE QUALITY IS A WHOLE DIFFERENT THING. IT CAN MEAN DIFFERENT THINGS DEPENDING ON THE PROCESS, LITIGATOR, IT ACTUALLY -- IT'S SORT OF LIKE ART. IF YOU LIKE IT OR YOU DOAN. BUT THE REALITY OF QUALITY IN THIS SENSE IS WHAT WE WANT IS PRECISION PATENTING TO OCCUR. SO IF WE'RE GIVING AN TIME, MOST OF THE TIME SHOULD BE ON THE ANALYSIS OF THE QUALITY IN TERMS OF THE CLAIM CONSTRUCTION AND USE OF THOSE WORDS AT THE END OF A PATENT THAT ARE CLAIMS, SO ONE, TO ENSURE THAT NO OTHER ART EXISTS THAT MAKES IT UNPATENTABLE, AND TWO, THAT THE TERMINOLOGY IS ABLE TO PASS OUR TEST OF STANDARDS. SO IN THIS SENSE, THAT'S HOW WE DEFINE QUALITY. QUALITY IS BEING ABLE TO SPEND MORE TIME FULLY UNDERSTANDING WHAT IS IN THE ACTUAL PATENT. SO, YOU KNOW, TO TAKE IT UP ANOTHER LEVEL, YOU KNOW, I LISTENED TO LAST NIGHT ACTUAL SUPREME COURT JUSTICE SPEAK ABOUT HOW THEY SPEND A LOT OF TIME TRYING TO UNDERSTAND AND SPLIT APART THE MEANINGS OF WORDS. WE DO THAT BASICALLY ON A VERY COMPRESSED SCHEDULE TO REALLY CHALLENGE WORDS AND UNDERSTANDING OF IT. THE BETTER WE GET AT THAT EARLY ON, THE IMPROVEMENT OF QUALITY WILL BE SEEN LONG-TERM. BUT WHAT WE'RE FIGHTING UP AGAINST NOW IS THE TIME BEING SPENT SEARCHING, THE TIME BEING SPENT WRITING AND NOT TO ANALYZE. SPEAKER: A BRIEF POINT TO HIGHLIGHT WHAT YOU SAID NEEDS TO BE PUT INTO COMMERCE AS A WHOLE AND PROBABLY OUR INDUSTRY AS A WHOLE, YOU'VE WELL ABOUT EFFICIENCY AND WHY THE BEST TECHNOLOGY OUT THERE REQUIRE US TO THINK IN A SOPHISTICATED WAY ABOUT THE TRADE-OFFS WE ARE MAKING, SO I WOULD COMMEND YOU FOR THAT BECAUSE THIS IS REALLY IMPORTANT AND I THINK IT'S SOMETHING TO KEEP IN MIND ON EVERYTHING WE DO TECHNOLOGIES THAT MANY OF US ARE IN LOVE WITH CAN PUSH US TO -- ARE INVOLVED WITH CAN PUSH US TO CONSISTENTLY HOLD ONTO THAT MINDSET, WE'LL BE PRESSURED BY A PROCESSES TO MOVE TOWARD EFFICIENCY WHEN THAT'S NOT THE BEST THING FOR THE AGENCY, FOR THE COMMUNITY, FOR THE POPULATION, AND FOR OUR USERS, SO I WANTED TO COMMEND YOU FOR THAT STATEMENT AND SAY THAT PLEASE MAKE SURE THAT LITTLE LO I CAN PERVADES ACROSS COMMERCE BECAUSE WE NEED IT. SPEAKER: THANK YOU VERY MUCH. I APPRECIATE IT. SPEAKER: THAT'S MUCH. TOM: THANKS SO MUCH SPEAKER: REALLY PRESH IT. NOW WE HAVE ED KEARNS, WHO IS THE -- CAN I CALL YOU THE CHIEF DATA OFFICER AT NOAA NOW, ED? SOON, ACTING. GREAT. OUR LEAR ON THE DATA WORK AT NOAA AND HAS BEEN A GREAT PARTNER TO AWFUL US AS WE'VE BEEN MOVING FORWARD ON OUR THANK YOU VERY MUCH. ABOUT A YEAR AGO WE BRIEFED YOU ON THIS PROJECT IN BOULDER, I'LL TRY ON GET TO THE RESULTS AND QUICKLY GET TO THE DISCUSSION ABOUT SOME OF THE THINGS WE'VE BEEN DISCOVERING. SO SECRETARY PRITZKER ANNOUNCED THIS IN APRIL 15, IF YOU RECALL, AND I LIKE TO KIND OF POSE IT AS A GENERAL CHALLENGE BETWEEN THE PORTAL APPROACH VERSUS A PLATFORM APPROACH, RIGHT? SO RIGHT NOW NOAA HAS A LOT OF PORTALS FROM WHICH YOU CAN COME AND GET DATA, BUT THIS SAYS A FUNDAMENTALLY DIFFERENT APPROACH COMMERCE TO PUT THE DATA OUT IN A WAY THAT THE DATA MAY BE MORE USABLE WITHIN THESE PLATFORMS AND OF COURSE AT THE HEART OF THIS, THIS IS A BUSINESS EXPERIMENT. WE'RE NOT TALKING ABOUT A TECHNICAL STUFF, YOU'VE HEARD A LOT OF TECHNICAL DISCUSSIONS. THIS IS REALLY MORE ON A BUSINESS CHALLENGE THAN IT IS A TECHNICAL CHALLENGE CHALLENGE. SO BRIAN A YEAR AGO MENTIONED WE WERE STARTING THIS PROJECT WITH THE NEXT ROD WEATHER RADAR DATA, SO WE HAVE A YEAR'S WORTH OF MET TRICKS NOW THAT -- METRICS NOW THAT HAVE BEEN COLLECTED AFTER THIS DATA HAVE BEEN MADE AVAILABLE ON AMAZON'S WEB PLATFORM, AND IN THIS GRAPH I WON'T SPEND A LOT OF TIME ON IT, BUT YOU CAN SEE IN THE BLUE IS THE NOAA METRICS FROM THE NATIONAL CENTERS FOR ENVIRONMENTAL INFORMATION, THE VOLUME OF DATA FROM THE WEATHER RADAR BEING ACCESSED IN TOTAL. WE SEE THAT RED LINE WHERE THE AMAZON WEB SERVICE WAS ANNOUNCED EXACTLY A YEAR AGO, AND YOU'LL SEE IN THE GREEN THOSE VOLUMES THAT HAVE BEEN AK CELSIUSED THROUGH AMAZON ITSELF. SO YOU CAN SEE -- ACCESSED THROUGH AMAZON ITSELF. SO YOU CAN SEE THERE'S BEEN A BIG CHANGE IN OUR USAGE PATTERNS. I HAVE SOME NUMBERS TO SHOW WHAT THOSE ARE. INCREASE OF 130% OF JUST HOW MUCH OF THE DATA ARE BEING USED. RAW DATA THAT NOAA IS RECEIVING RIGHT NOW IS BEING SERVED THROUGH AWS, NOT THROUGH NOAA. ABOUT 64% OF THE DATA ARE STAYING ON THE AWS PLATFORM, SO THE AWS, IF YOU COME AND USE DATA ON THE PLATFORM, THE METER RUNS. IF YOU WOULD LIKE TO EXTRACT THE DATA FROM AMAZON, THEY'LL LET YOU DO SO FREELY, SO TO THEIR OWN SYSTEMS, BUT A LOT OF PEOPLE, 64%, ARE LEAVING THEM SINGLE POINT OF ACCESS NOW FOR REALTIME OR HISTORICAL DATA, THAT WASN'T TRUE BEFORE. THE SPEED WITH WHICH PEOPLE CAN DO WORK IS NOW GREATLY INCREASED . CLIMATE CORP WAS ONE OF THE PARTNERS OF AMAZON GOING INTO THIS PROJECT AND THEY WERE ABLE TO DO IN A FEW DAYS THAT HAD TAKEN NOAA YEARS TO ACTUALLY PROCESS THAT MUCH DATA, SO IT WAS A VERY BIG CHANGE IN HOW FAST THE WORK CAN GET DONE, THE ANALYSIS CAN GET DONE. AND CHEAP ARE. THE LOADS ON NOAA'S ARCHIVE SYSTEMS ARE DOWNL OVER 50%, SO THAT'S AGAIN MEASURABLE AND OVER TIME THAT COULD BECOME A COST SAVING FOR NOAA. THERE'S ALSO NEW SCIENCES COME UP, SO THERE'S BEEN BIRD MIGRATION, INSECT STUDIES THAT HAVE POPPED UP THAT ARE NOVEL APPLICATIONS THAT WEREN'T POSSIBLE BEFORE AND OF COURSE ALL THIS HAS BEEN DONE AT NO NET COST TO THE U.S. TAXPAYER. SO THERE'S A PAPER COMING OUT IN MORE DETAICIALTION ALL THE DETAILS ON THIS SOON IN THE BULLETIN OF THE AMS. EXCUSE THERE'S ABOUT TEN OTHER DATA SETSZ THAT WE'RE ACTIVELY WORKING WITH THE COLLABORATORS ON, THE STATIONARY SATELLITE THAT'S ABOUT TO GO UP NEXT MONTH MONTH, FIDUCIARYIES CATCH DATA WEATHER PREDICTION HAS BEEN DISCUSSED IN STAGE I OF THIS PROJECT, VARIOUS CLIMATE DATA THAT ARE AVAILABLE AND OTHER PRODUCTS ACROSS NOAA. WHAT WE'RE RUNNING FAST NOW IS TRYING MATURITY, WE HAVE A EXPWREER A HALF LEFT IN THE INITIAL PERIOD, WE'RE TRYING TO ASSESS LIKE WE DID FOR AMAZON WHETHER RADAR DATA, TRYING TO A ASSESS THE METRICS TO SEE WHAT THE RESULTS ARE FOR DOING WORK THIS WAY. DOCUMENT THOSE AND REALLY START PLANNING NOW FOR WHAT'S GOING TO COME AFTER THIS EXPERIMENTAL PHASE, WHAT WE'RE CAWMG THE POST THINGS WE WOULD LIKE TO TALK ABOUT AND TALKING WITH HEATHER AND BILL EARLIER ABOUT THIS, WE WANT TO REALLY FOCUS ON WHAT WE'VE LEARNED. SO ONE OF THE THINGS IS, YOU KNOW, MOST OF NOAA'S DATA ALMOST ALL OF NOAA'S DATA REALLY ARE AVAILABLE NOW, OPEN AND FREELY, IF YOU KNOW WHERE TO GO AND WHO TO ASK, RIGHT? BUT EVEN THOUGH DATA THAT'S AVAILABLE RIGHT NOW THAT ARE ON PUBLIC-FACING WEB SERVICES, THEY'RE NOT ALWAYS MOVING. WHY AREN'T THEY MOVING? UNDERSTAND THEM. THAT'S A VERY, VERY BIG THING. THE COMMUNITY THAT'S TRYING TO DOWNLOAD THE DATA AND USE THE DATA HAVE TO UNDERSTAND THEM, MANY OF THEM ARE VERY TECHNICAL, VERY HARD TO DO. THAT DISCUSSION WITH THE NOAA EXPERTS IS A KEY PART THAT WE'VE DISCOVERED. TO GET THE DATA TO MOVE WITHIN THE CRADA PROJECT, THERE NEEDS TO BE SOMEBODY ELSE IN THE DISCUSSION, COLLABORATOR, THERE NEEDS TO BE A THIRD PARTY THAT OWNS THE PROBLEM, THAT HOLDS THE PROBLEM THAT'S TRYING TO BE SOLVED. AND CORP THAT GOT THE INITIAL DISCLOSURE, AND THEN A LOT OF PEOPLE CAME TO THE AMAZON PLATFORM, ECOSYSTEM OF USERS GREW AROUND THE DATASET AVAILABLE. CLIMATE CORP INITIATED THAT. YOU NEED TO BRING A BUSINESS MODEL. ALL THE OWN INVESTMENT ON THE TABLE, OWN MONEY IN THE GAME AND THEY WANT TO MAKE SURE BEFORE THEY START SO INVEST A LOT OF TIME AND RESOURCES INTO THE DATASET, THAT FAIR. SO HOW CAN THOSE NEW USERS AND MARKTZ BE IDENTIFIED? PEARTSZ INTO THE CONVERSATION? THAT'S A BIG CHALLENGE. NOAA SERVES ITS EXISTING COMMUNITIES IT KNOWS ABOUT LIKE THE WEATHER ENTERPRISE, ALREADY SERVING THAT WELL, BUT FOR OTHER USES OF THE WEATHER DATA BEYOND THE THAT COMMUNITY, HOW DO WE BRING IN THIRD PARTIES WITH THE COLLABORATORS AND TRYING TO SET UP THOSE DIALOGUES HAS BEEN A CHALLENGE. DATA QUALITY. SO AS I HAVE HERE, DANA TALKED ABOUT THEARLER, DATA QUALITY IS A BIG ISSUE FOR NOAA, AS YOU HEARD, YES, WE DO CARE ABOUT DATA QUALITY, DATA INTEGRITY, AND ONCE THE DATA LEAVES THE NOAA SYSTEM, DOES THAT RESPONSIBILITY FOR ENSURING THE DATA QUALITY END? SO THE LIABILITY ENDS, BUT GAME ON THIS AND WE STILL WANT TO ENSURE THE DATA QUALITY. AND TALKING MORE ABOUT WIP HE WILL MANIPULATION EARLIER BEFORE, BUT THAT WE CAN VERIFY TO, SAY, AN ANALYTICAL TOOL THAT IS BEYOND OUR ABILITY TO, YOU KNOW, CONTROL, AND THE EXAMPLE I'LL USE WILL BE GOOGLE, SO GOOGLE TOOK A LOT OF CLIMATE DATA AND PUT INTO BIG QUERY AS PART OF THE CLIMATE PREP PROJECT JUST ANNOUNCED BY THE WHITE HOUSE AT THE END OF SEPTEMBER AND THE NATIONAL CENTERS FOR ENVIRONMENTAL INFORMATION WERE ABLE TO VERIFY THE TRANSFER OF THE DATA THAT THEY'RE VALID AND HIGH QUALITY INTO GOOGLE BUT ONCE THEY LOAD IT INTO BIG QUERY QUERY, THEN HOW DO WE DO T BECAUSE THEY JUST CHANGED -- THEY CHANGE THE FORMAT OF THE DATA, THEY HAVEN'T CHANGED THE DATA, BUT HOW DO WE VERIFY THAT WITH THEM AS PARTNERS? BECAUSE AGAIN, IT MAY NOT BE WILLFUL MANIPULATION, IT JUST IS THE FACT MAYBE THEY PUT IT IN WRONG OR GOT THE REFERENCE GOT MESSED UP OR SOMETHING LIKE THAT. WE HAVE TO FIND WAYS AND EXPLORE WAYS THAT WE CAN ENSURE THE QUALITY THROUGHOUT THE QUALITY CHAIN OF THE DATA. HOW CAN NOAA NOAA'S EXPERTISE BE BEST LEVERAGEED? THAT'S THE REAL ASSET THAT NOAA HOLDS. IT'S NOT 16 PETABYTES OF DATA AND OTHER 100 PETABYTES OF MODEL RESOURCE, BUT THE REAL RESOURCE I'VE BEEN SAYING WITHIN NOAA ARE SO THE DATA CAN BE USED. OKAY. THIS IS MY LAST SLIDE. YES. MY LAST BULLET ON MY LAST SLIDE, SO I'M RIGHT ON TIME. I DID THIS TALK IN THREE MINUTES AT THE CONFERENCE CENTER THE OTHER WEEK WEEK, SO I COULD GO FASTER IF YOU WANT. (LAUGHTER). BUT YOU SEE, LIGHTNING TALK IN NOAA YOU HAVE TO BE CAREFUL BECAUSE YOU'LL THINK I'M TALKING ABOUT LIGHTNING. NO LIGHTNING. REALLY THAT PLATFORM VERSUS PORTALS APPROACH. WE'RE SEEING IT WITH THE NEXT DATA, THE WEATHER DATA ON AMAZON, IS IT REALLY MAKING THE DATA MORE USABLE OR NOT? I'VE BEEN SAYING PROBABLY LOWER BOUND IN THE POSSIBILITY, IT'S A VERY OBSCURE FAMILIAR WITH IT AND PART OF THE EXPERT COMUN, IT'S HARD TO USE, USE -- COMMUNITY, IT'S HARD TO USE, AND WE DON'T HAVE VISIBILITY INTO THAT NEXT STEP BEYOND THE INTEMPTIVE LAYER AROUND AMAZON, WE DON'T KNOW THE INTERMEDIARYIES ARE CARRYING ON, THE VALUE WITHIN THE RADAR DATA INTO THE COMMUNITY, BEYOND THAT WE DON'T HAVE A GOOD INSIGHT INTO THAT. BUT I'M HAPPY TO TAKE YOUR QUESTIONS. SPEAKER: ED, THAT WAS TERRIFIC. WHY DON'T I TURN IT OVER TO BILL AND HEATHER. BILL: OKAY. I REALLY WANT TO COMMEND NOAA ON I THINK A COUPLE OF KEY THINGS HERE. ONE IS SORT OF THE SANDBOX APPROACH THAT YOU'VE TAKEN TO EXPLORE BUSINESS PROBLEM WHERE YOU DIDN'T KNOW THE ANSWER AND SO YOU WENT TO AN UNSTRUCTURED PROJECT THAT WAS STRUCTURED TO GET YOU TO WHATEVER, I THINK YOU'VE DONE THAT WELL. THE SECOND IS TO BUILD ON THE ECOSYSTEM OF OTHER PART PARTNERS THAT ARE NEEDED TO GET DATA OUT INTO THE WORLD, AND BOTH OF THOSE THINGS HAVE EMERGEED I THINK AS SUCCESS STORIES OF WHAT YOU'VE DONE. BUT I WAS HOPING YOU COULD MAYBE EXPAND ON THAT AND TALK ABOUT HOW THOSE SUCCESS STORIES WITHIN NOAA MIGHT TRANSFER OR NOT TRANSFER TO OTHER PARTS OF COMMERCE. ED: YES, SO WE'RE BUILDING PRIDGES BETWEEN NOAA AND COMMERCE, SOME OF THE ACTIVITIES YOU'VE HEARD ABOUT TODAY, THE IDEA OF HAVING THESE PARTNERSHIPS THAT ARE NOT JUST STRAIGHT-UP CONTRACTS, A FEE FOR SERVICE KIND OF THING, I ALREADY SPREADING ACROSS COMMERCE, THE FACT THAT, YES, WE THE OUTCOMES OF THIS PROJECT WOULD BE IDENTICAL IDENTITY -- WOULD BE HIGHLY TECHNICAL DATA, HIGHLY TECHNICAL PEOPLE, HOW YOU BRING IT INTO THE PARTNERSHIP, EVERYBODY ACROSS COMMERCE WILL LEARN HOW THAT WORKS OR NOT, THE QUITE FRANKLY FRANKLY. SPEAKER: I WANT TO ALSO, FOR ANOTHER ASPECT AS AN ORGANIZATION THAT ADVOCATES FOR OPEN DATA AND SUPPORT FOR OPEN DATA PROJECTS, YOU'VE GIVEN US THIS BEAUTIFUL SUCCESS STORY THAT ALLOWS US TO USE THE WORDS LEVERAGE, THE TAXPAYER INVESTMENT, AND RECH, RESEARCH IN A BEAUTIFUL WAY. WE TALKED A LITTLE BIT ABOUT THE APPLICABILITY OF THIS PROJECT ACROSS JUST A VARIETY OF DIFFERENT SECTORS, WHICH I THINK THE QUESTIONS THAT WE AS A COMMITTEE KEEP ASKING OURSELVES IS WHAT CAN WE DO? HOW CAN WE AS A COMMITTEE HELP YOU TO LEVERAGE THIS SUCCESS STORY AND KIND OF MAYBE AMPLIFY HOW IT CAN SECTORS? SPEAKER: ONE THING I WOULD BE VERY HAPPY TO HEARP THE BRING MAYBE MORE. GENERAL ANALYTICS CAPABILITIES THAT ARE PRESENT NOW, THROWFT THE COLLABORATORS THAT WE HAVE, YES, FOCUSED ON SORT. VERY BASIC INFRASTRUCTURE AS A SERVICE APPROACH KIND OF THING, BUT THE VALUE CHAIN, OF COURSE, WE KNOW GOES THROUGH ANALYTICAL AND NEW AI TOOLS AND STUFF LIKE THAT, AND WE HAVEN'T HAD A LOT OF SUCCESS IN PLUGGING THOSE INTO THE BUSINESS MODELS RIGHT NOW, SO THAT WOULD BE VERY WELCOME. ALSO ADVICE ON HOW TO BUILD EVEN A BUSINESS MODEL AROUND A BIG DATA APPROACH BECAUSE AS YOU'LL NOTICE, WHAT I MENTIONED, A LOT OF THESE DATA SETSZ WE'RE WORKING ON ARE LIKE SINGLE LARGE DATA SETSZ, NOT ACTUALLY AN AGGREGATION OF A BUNCH OF SMALLER DATASETS, WHICH IS REALLY CLOSER TO WHAT BIG DATA IS ON THESE KIND OF APPROACHES. SO AS WE'VE FOUND, IT'S VERY HARD GOING ON OUR COLLABORATORS AND TRYING TO CONSTRUCT A BUSINESS CASE FOR SOMETHING THAT MIGHT SEE SOME COOL RESULTS, SO IF YOU'VE GOT ANY ADVICE FOR HOW IT WOULD BE VERY MUCH APPRECIATED. SPEAKER: HOW YOU ENSURE QUALITY DOWN THE LINE, I THINK THE FIND THAT I FIND MOST USEFUL IS WORK DONE BY ANN WASHINGTON WHERE SHE ARGUES WE SHOULD BE THINKING OF THESE AS DATA SUPPLY CHAINS, AND I THINK THAT'S A REALLY INTERESTING JUXTAPOSITION AGAINST THE OPEN DATA RHETORIC WE OFTEN HEAR BECAUSE ORCH WHAT IT MEANS, WE NEED OPEN, HAVE FUN WITH IT, WHATEVER HAPPENS IS FINE, ACTUALLY I THINK YOUR POINT, AND THIS IS A UNIQUE PLACE WHERE YOU CAN ACTUALLY PROVIDE LEADERSHIP IS TO REALLY THINK ABOUT HOW DO WE ENSURE QUALITY ACROSS THE ENTIRE SUPPLY CHAIN OF DATA, ALL IT MAY GET USED SO THAT WE CAN ACTUALLY ENABLE THE KIND OF COMMERCIAL AWESOMENESS THAT WE MIGHT IMAGINE, AND I WOULD SAY THAT THIS HAS BEEN A CHALLENGE IN MANUFACTURING WHERE ONE OF THE THINGS THAT CAME TO BE WAS EVERYBODY REALIZED ACTUALLY WE NEED TO MAKE SURE THE SUPPLY CHAIN IS RESPONSIBLE, THAT WE'RE MATERIALS, THAT WE'RE GETTING THE LABOR IS ACCEPTABLE, ET CETERA, ET CETERA, I THINK IT'S A UNIQUE OPPORTUNITY, ESPECIALLY ARE FOR THINKING ABOUT DATA AS A COMMERCIAL COMMERCIAL TOOL TORQUES THINK ABOUT IT IN SUPPLY CHAIN METAPHORS FOR EVERYBODY ELSE TO BE RESPONSIBLE FOR THE SUPPLY CHAIN. ED: I THINK THE QUALITY THE VALUE OF VAGUE HAVING YOUR DATA TRACEABLE TO, SAY, A NOAA STANDARD COULD BE FOR A CONSUMER THINK THE COMMENT EARLIER, THE KNEELED OF THE FEDERAL AGENCIES TORN WHERE THESE ARE GOING, SO ONCE THEY'VE GONE INTO SOME COLLABORATORS' TOOLBOX AND FURTHER DOWN THE SUPPLY CHAIN, HOW DO WE KNOW, AND BE VERY OPEN OPEN, I WOULD BE VERY OPEN TO YOUR ADVICE, TOO, ON HOW WE WOULD CONSTRUCT THESE KIND OF OPEN REGISTRIES. WE'RE CONSIDERING THINGS LIKE BLOCK CHAIN AND OTHER TECHNOLOGIES THAT WE COULD USE PROVIDED OPEN REGISTRY FOR OPEN DATA SO PEOPLE THAT ARE USING THE DATA CAN LET US KNOW THEY'RE USING THE DATA AND THEY CAN KNOW WHERE THE DATA CAME FROM AND IF WE CAN BUILD SOME DATA SECURITY INTO THAT AT THE SAME TIME, AND THIS IS NOT SYSTEM SECURITY THAT HAS BEEN A CHALLENGE, RIGHT, FOR A LOT OF THINGS, BUT THIS IS DATA SECURITY JUST TO ENSURE THAT THE DATA THROUGHOUT THE SUPPLY CHAIN ARE STILL AS VALID AT THE EN OF THE SUPPLY CHAIN AS THEY WERE AT YOU'VE GOT ANY IDEAS, WE'RE VERY OPEN TO SUGGESTIONS. WE KNOW THIS IS A RESEARCH ACTIVITY. SPEAKER: ANYBODY ELSE? SPEAKER SPEAKER: I CAN SEE YOUR PROBLEM OR GOOGLE WHERE YOU'RE PARTNER PARTNERING WITH THEM TO PROVIDE DATA TO END USERS AND YOU WOULD WANT TO MAKE SURE, THAT DOES SEEM MORE PLIEK A UNIT TEST, I'M SURE WHEN BAY UNIT TEST LEAVES YOUR ORGANIZATION, IT SEEMS LIKE TO THEN RECHECK IT ONCE IT'S BEEN REFORMATTED IN THE EXAMPLE YOU GAVE ON GOOGLE'S PLATFORM YOU WOULD JUST READ CHECKS AND BASIC KIND OF 20 TO 100 METRICS AND MAKE SURE THAT YOU COULD COMPUTE THEM THE SAME WAY, I GUESS, SO THAT SEEMS KIND OF SOLVABLE. THE QUESTION I HAVE, A DEEPER QUESTION, WAS SOME OF THE THINGS YOU DESCRIBED ABOUT KIND OF MAKING BUSINESS CASES FOR THINGS FOR THE DATA, IS THAT BECAUSE, I MEAN, AND ALSO THE NEED FOR ANALYTIC CAPABILITIES ON TOP OF JUST THE DATA PROVISIONING THAT YOU'RE ALREADY USERS, 64% OF THESE USERS THAT ARE TAKING THE DATA OFF AND USING IT, THEY'RE DOING ALL THAT THAT, DOING BEST AN AN LIT, DO YOU THINK YOU NEED TO DO SOME PROOF OF CONCEPTS ABOUT HOW YOU COULD USE THE DATA OR WHAT IS THAT ABOUT? ED: EXACTLY, BECAUSE IF WE UNDERSTOOD THAT SOME OF THESE DATASETS THAT WE'VE PROPOSED AND TALKED ABOUT WITH OUR COLLABORATORS, IF WE UNDERSTOOD HOW THEY COULD BE EMPLOYED BY THOSE OTHERS, SO WITH THE WEATHER RADAR WE WERE ABLE TO SAY, BECAUSE THERE'S AN ACTIVE COMMUNITY THAT TRIED TO FOR MANY YEARS DOWNLOAD THE ENTIRE DATASET FROM NOAA AND WEREN'T ABLE TO DO SO, WE'RE ABLE TO SAY IF YOU DO THIS, THESE PEOPLE WILL COME, AND SOME OF THEM SPOKE UP AND SAID YES, WE'LL USE THIS PLATFORM AND OFF THEY WENL. BUT FOR OTHER DATASETS IT'S NOT QUITE AS APPARENT, SPECIFICALLY FOR THE OTHER COMMUNITIES THAT MAY WANT TO USE THE DATA IN NOVEL WAYS, AND TO ACTUALLY GET THE DATA OUT THERE ONTO THE PLATFORM IS THE FIRST NECESSARY STEP AND THAT'S WHERE WE'RE STRUGGLING WITH THE BUSINESS CASE BECAUSE IN ORDER TO ENABLE THAT INVESTMENT OF RESOURCES AND LABOR TO DO THAT ON THE COLLABORATOR'S SIDE, THEY HAVE TO HAVE A CLEAR PICTURE OF WHAT THOSE POSSIBLE USES ARE, AND RIGHT NOW IT'S PREURCH A CHALLENGE TO DO THAT. -- PREURCH SPEAKER: YOUR CHART WHERE YOU SHOW AWS USAGE LED ME TO BELIEVE RETRIEVE OFF THE NOAA ARCHIVE WAS SIMILAR TO WHAT I COULD RETRIEVE OFF AWS. IS THAT TRUE? ED: EXACT SAME, BEEN VERIFIED, IT'S THE EXACT SAME DATA. ANECDOTALLY ONE OF THE MEASURES OF SUCCESS TOO ARE NOAA'S OWN SIGN TESTS GOING TO GET THE DATA FROM AMAZON OR -- YEAH, ANECDOTALLY I CAN TELL YOU, FROM SCIENTISTS IN THE BUILDING LITERALLY SITTING ON TOP OF THE ARCHIVE THAT ARE GETTING THEIR DATA FROM AMAZON INSTEAD OF ARCHIVE WITHIN THE BUILDING BECAUSE THEY KNOW IT'S EXACTLY THE SAME DATA AND IT'S FASTER. SPEAKER: OKAY, SO THAT IT'S FASTER. OKAY. SPEAKER: CAPSTONE PERSPECTIVE ON THIS FABT STICK WORK AND ALSO CONNECTED TO THE SUPPLY CHAIN CONCEPT. IN OUR FIRST MEETING WE TALKED ABOUT A DIGITAL WATERMARK, HOW DO WE UNDERSTAND DOWNSTREAM CONSUMPTION SO WE CAN OPT MILES PER HOUR THE SUPPLY CHAIN. THERE IS NO BRANDING TRACKER DOWNSTREAM, IT WAS OPEN, WE DON'T REQUIRE A LOG IN TO ACCESS THE DATA GENERALLY. IF YOU MPG THE WHOLE SYSTEM FOR DATA, I THINK THAT'S STILL A MOONSHOT OPPORTUNITY FOR US FROM A TECHNICAL PERSPECTIVE I HAVE PERSPECTIVE. HERE IS THE BUSINESS, NETFLIX OR AMAZON, IF YOU LIKE THIS DATA, YOU'LL ALSO LIKE THIS DATA. WE DON'T HAVE THAT BECAUSE WE DON'T ACTUALLY KNOW WHAT FOLKS ARE NECESSARILY USING. FROM A PATENT EXAMINER'S PERSPECTIVE, IF YOU'RE LOOKING AT THIS APPLICATION, YOU SHOULD PROBABLY IT, SO THIS IS THAT SYSTEMIC GAP THAT I THINK REALLY CHALLENGES THE ENTIRE SECTOR AND I THINK A LOT OF WHAT THIS FANTASTIC EXPERIMENT HAS TRIED TO CREATE IS AT LEAST SOME DEGREE OF TRACEABILITY. NOT JUST FROM A BUSINESS PERSPECTIVE BUT ALSO, OKAY, WELL F WE PARTNER WITH OTHER FOLKS, CAN IT HELP US FIGURE OUT HOW WELL IT'S BEING USED SO WE CAN CIRCLE BACK ON THIS DATA AND FIND MORE EFFECTIVE WAYS OF DELIVERING IT OUT TO THE RIGHT CUSTOMERS IN THE RIGHT WAY. SO AGAIN, IT'S SOMETHING I THINK WE'VE BEEN TRUG STRUGGLING WITH SINCE THE BEGINNING, I THINK THIS EVOLUTION HAS DEMONSTRATED EXAMPLES AND I THINK IT'S STILL ONE OF THE MOST SYSTEMIC RISKS AND OPPORTUNITIES WE HAVE AS A DEPARTMENT, AS A SECTOR, AS A COUNCIL. LY. >> SPEAKER: YOU SKIPPED OVER ME ME. (LAUGHTER). SPEAKER: I'M NOT TRYING TO. SPEAKER: THE FIRST QUESTION THERE ABOUT DEMAND AND BUILDING NEW MARKETS. LET'S SAY THAT THERE ARE KIND OF THREE WAYS TO ATTRACT THOSE NEW USERS, SO THE FIRST ONE IS TO MAKE THE DATA VISIBLE, IN OTHER WORDS, THIS DATA EXISTS, LET IT INSPIRE YOU, WHICH IS KIND OF WHAT DANA WAS MENTIONING, MAYBE IT INSPIRES SOME PEOPLE. THE SECOND THING IS THAT THERE ARE PEOPLE OUT THERE WHO HAVE A PARTICULAR PROBLEM TO SOLVE, AND THEY NEED TO BE ABLE TO FIND YOU WHEN THEY'RE LOOKING FOR THE THING WHICH IS NECESSARY. SO THIS IS A I HAVE IDENTIFIED A DEMAND ALREADY IN THE SYSTEM AND YOU BEING ABLE TO DO IT. THE THIRD ONE IS ACTUALLY GENERATEDING OR IDENTIFYING DEMAND AND THEN GETTING IT IN FRONT OF THE PEOPLE WHO WOULD ACTUALLY TRY SOLVE THAT DEMAND. THIS MIGHT BE A PLACE WHERE YOU ACTUALLY COULD DO MORE. SO I'M SURE, I ASSUME, THAT NOAA GETS A BUNCH OF REQUESTS FROM PEOPLE IN THE WORLD SAYING, HEY, DO YOU HAVE X X? WE DON'T, WE, NOAA, DON'T HAVE THE RORTSDZ, BUT THERE ARE SOME PEOPLE OUT THERE WHO MIGHT BE INTERESTED IN PLAYING AROUND AND DEVELOPING X. IT WOULD BE THE EQUIVALENT OF KIND OF AN RFP WOULD SORT OF REACH OUT TO YOU AND SAY, HEY, I'M TRYING TO SOLVE WHATEVER, THEN THE IDEA SU NEED TO BE ABLE TO PUSH THAT DEMAND BACK INTO A MARKETPLACE OF POTENTIAL USERS OF THE DATA. THINKING ABOUT MORE OF WHAT THAT WOULD LOOK LIKE, BUT THE MAIN THING IS TRY TO IDENTIFY DEMAND, THEY KNOW THEIR WORK PRODUCT WILL BE USED AND MAYBE EVEN PAID SPEAKER: GREAT POINT. THAT PROCESS EXISTS IN MANY PLAIPSZ IN NOAA BECAUSE IT'S A CUSTOMER SERVICE FACING ORGANIZATION, MOST OF THE REQUIREMENTS GATHER GATHERED FROM THE USER COMMUNITY FUTURE INVESTMENTS AND STUFF LIKE THAT, THERE'S NOT AN EFFECTIVE COMMUNICATION RIGHT NOW OF THOSE REQUIREMENTS TO COMMERCIAL SECTOR RIGHT NOW, SO IT'S KIND OF A -- LIKE IT'S COOKED DOWN ON NOAA BUT DOESN'T MAYBE GET SHARED EFFECTIVELY RIGHT NOW WITH THE COMMERCIAL SIDE. SPEAKER: JUST QUICK FOLLOW-UP ON IT. THERE ARE SOME ORGANIZATIONS THAT EXIST OUT THERE WHICH SPECIALIZE IN IDENTIFYING POTENTIAL PARTICIPANTS, SPENT SOME TIME LAST WEEK WORKING ON AN X PRIZE DEFINITION AND X PRIZE IS REALLY PRE NURS WHO MIGHT ACTUALLY BE INTERESTED IN TAKING ADVANTAGE OF SOMETHING, IN THEIR CASE THE PRIZE IS ACTUALLY AVAILABLE TO DO IT. THE WAY THEY DO THAT IS BUILD GROUPS OF PEOPLE, NETWORKS PLUGGED IN, YOU WOULD BE A GREAT EXAMPLE, WHO ARE PLUGGED INTO THAT ENVIRONMENT AND THEN TAKE ADVANTAGE OF THAT AND THEN THEY BASICALLY, YOU SAID WE HAVE DEMAND FOR THESE THREE THINGS, THEY DO THE DISTRIBUTION FOR YOU YOU. SPEAKER: THAT'S A GLEAT POINT. ON THE SLIDE YOU SAW, I DUNLT MENTION IT IN MY COMENLTSES, BUT THE OCEAN DATA HAD BEEN MUCH HARDER TO MOVE THAN THE METEOROLOGICAL DATA, FOR EXAMPLE, BECAUSE THE MARKET OR WHATEVER, BUT AT THE MARINE ALLIANCE MEETING UP IN SAN DIEGO NEXT MONTH THERE IS INDEED AN X PRIZE BEING ANNOUNCED ON USE OF OCEAN DATA. EXACTLY. GREAT IDEA. SPEAKER: ALAN, I MIGHT ADD, I WAS SMILING AS YOU WERE TALKING BECAUSE ONE OF THE THINGS WE'VE BEEN THINKING ABOUT WITH ALL OF OUR SENIOR CAREER LEADERSHIP IS EXACTLY WHAT YOU HAVE BEEN JUST SAYING, AND WE'RE GOING TO TALK ABOUT IT THIS AFTERNOON. I WAS LOOKING AT MY COLLEAGUE LAURA McGORMAN WHO TOGETHER WITH NANCY PORTAC THE DEPUTY DIRECTOR AT CENSUS WILL BE LEADING A WORKING GROUP ON ISSUES GOING FORWARD ON ISSUES LIKE THAT, HOW DO YOU ENGAGE, DIRECTOR OF THE CENSUS JOHN THOMPSON WHO HAS BEEN VERY SUPPORTIVE OF THIS KIND OF ENGAGEMENT IS HERE TODAY SO I WELCOME THAT COMMENT REALLY BECAUSE WE'RE LOOK THINKING HARD GETTING DOWN IN THE WEEDS ON THAISH EWE. I KNOW I TALKED ABOUT THIS AT THE TOP OF THE MEETING, BUT PART OF OUR INSTITUTIONALIZATION GOAL IS TO HAVE A LOT OF THIS WORK THAT THE ED, BUREAUS AND AROUND THE DEPARTMENT LIKE ED AND OTHERS ARE REALLY CARRYING THE TORCH FORWARD AND WE'VE CREATED THIS DATA EXECUTIVE COUNCIL AND I WANTED TO HIGHLIGHT TOM BEACH ASK ED KEARNS ARE ACTUALLY LEADING THE TEAM ACROSS THE DEPARTMENT THAT ARE TRYING TO THINK ABOUT HOW DO WE INTERNALLY MEANINGFUL COORDINATION GOING FORWARD AND THEY'LL COME UP WITH BY OUR OWN LEADERSHIP RATHER THAN SOME TOP-DOWN APPROACH, AND BOTH LEADING THAT. WE HAVE A WORKING GROUP LED BY LAURA AND JEFF CHEN AND NANCY POTAC THINKING ABOUT THE BIG ISSUES AND THAT'S A LOT OF REALLY INTERESTING WORK THERE, AND ON THE TECHNOLOGY ACQUISITION CHRKS WE SPEND A LOT OF TIME THINKING ABOUT, ESPECIALLY THROUGH THIS NEW OPPORTUNITY ON THE JOINT VENTURE PROCESS, WE HAVE AVI BENDER, SHAM SUNDER AND OUR DEPUTY UNDERSECRETARY HERE BRAD BURK WHO HAS BEEN AN ABSOLUTELY CRITICAL MEMBER OF OUR LEADERSHIP TEAM WILL BE LEADING THAT WORK GROUP AS WELL. I APPRECIATE, ED, NOT ONLY YOUR WORK ON THIS BUT ON AN INTER INTERBUREAU VISION MUCH HOW WE ALL WORK -- OF HOW WE ALL WORK TOGETHER GOING FORWARD. OKAY. THANK YOU VERY MUCH. THAT WAS LIGHTNING ROUND TALKS. CAN I JUST HAVE A QUICK ROUND OF APPLAUSE FOR EVERYBODY WHO CAME UP HERE? (APPLAUSE) . THANK YOU ALL VERY MUCH. I'M GOING TO TAKE JUST A MINUTE BEFORE WE GO INTO THE DATA ACADEMY, NATASSJA, JUST ONE MINUTE, I NOTICE THAT NATALIE EVANS HARRIS, I'M SURE SHE'LL BE EXCITED THAT I'M JUST GOING TO CALL YOU UP HERE, BUT NATALIE IS HOUSE, SENIOR POLICY ADVISOR FOR OPEN DATA AND HAS BEEN LEADING THE DATA CABINET EFFORTS AND ALSO IS ONE OF THE MAIN FORCES BEHIND THE OPPORTUNITY PROJECT, AND SINCE I HAVE HER HERE, I'LL CALL YOU UP, NATALIE, IF YOU DON'T MIND SAYING A FEW WORE, I THINK IT'S IMPORTANT BECAUSE YOU MISSED IT, FROM OUR DATA ADVISORY COUNCIL SO HOW WE'RE INTERACTING WITH THE DATA CABINET, WHAT THE ROLE IS WITH THE OPPORTUNITY PROJECT AND OTHERS, SO IF YOU WOULDN'T MIND TAKING JUST A MINUTE OR SO TO INTRODUCE YOURSELF AND TALK ABOUT THOSE TWO EFFORTS AND OBVIOUSLY OUR MEMBERS WOULD BE HAPPY TO ASK YOU QUESTIONS. NATALIE: HELLO, EVERYBODY, NATALIE EVANS HARRIS, OSTP. AS YOU KNOW, THE ADD MIDGE STRAITION HAS BEEN VERY FOCUSED ON GETTING GOVERNMENT TO OPEN DATA AND MAXIMIZE THE VALUE OF THE DATA. STARTING AT THE BEGINNING OF THIS YEAR WE CREATED A DATA CABINET WHICH IS A COMMUNITY OF PRACTICE FOCUSED AROUND IDENTIFYING STRATEGIES, BEST PRACTICES, KEY TIPS, TRICKS AND TOOLS IN ORDER TO BE ABLE TO HELP AGENCIES TO DO JUST THAT, FIGURE OUT HOW TO BUILD THEIR DATA CAPACITY AND WHAT CAN BE DONE IN ORDER TO ADVANCE WHOLE OF GOVERNMENT THE WAY THAT WE'RE USING DATA BOTH WITHIN OUR AGENCIES AND ACROSS THE FEDERAL GOVERNMENT AGAINST OUR NATIONAL PRIORITIES SO WE STARTED WITH ABOUT 20CEO'S AT THE BEGINNING OF THIS YEAR ANN WAS ONE OF THEM SO INDIVIDUALS THAT MEET ON A MONTHLY BASIS TO DISCUSS THESE CHALLENGES, THE THREE KEY CHALLENGES THAT WERE IDENTIFIED AND WE CREATED WORKING GROUPS AROUND DATA TALENT WITH NATASSJA RIGHT-HAND WOMAN, SO WE HAVE THE TALENT WITH A FOCUS ON HOW DO WE PROFESSIONALIZE THE ROLE OF THE CDO AND HOW DO WE CREATE PATHWAY PATHWAYS FOR WHAT DATA TALENT CAN AND SHOULD BE DOING IN THE FEDERAL GOVERNMENT. SECOND WORKING GROUP WAS AROUND DATA RESOURCE MANAGEMENT. SO THAT DATA LIFE CYCLE AND HOW DO WE MAKE SURE THAT WE'RE RESPONSIBLY LEVERAGING DATA, NOT JUST OPENING IT UP, BUT MAKING SURE THAT WE'RE ACTUALLY COLLECTING AND MANAGING AND PROTECTING IT PROPERLY. THEN THE THIRD WORKING GROUP IS AROUND WHAT WE'RE CALLING DATA POLICY. SO WHAT THEY DID WAS THEY CREATEED THIS MATURITY MODEL WHICH DEFINES FEDERAL DATA CAPACITY ACROSS SIX DIFFERENT LANES, SO NOT TAKING IT OUT OF JUST THE IT TALKING ABOUT THE GOVERNANCE OF T THE ANALYTICS CAPABILITIES, THE TALENT, THE SYSTEMS TECHNOLOGY AND ALL THOSE DIFFERENT THINGS AND THEY CREATED A SELF-ASSESSMENT TOOL FOR DATA LEADERSHIP TO BE ABLE TO ASSESS WHERE THEY ARE CAPACITY-WIDE AND WORK TOWARDS WHERE THEY WANT TO GO IN THE FUTD AND HOW DO THEY WANT TO GET THERE. SO WE ARE MOVING FORWARD FORWARD, WE'VE IDENTIFIED THESE TOOLS I'VE SHARED WITH THE GROUP GROUP. IF YOU'RE INTERESTED. IN IN ADDITION TO DOING THAT, DATA SCIENCE SUBCOMMITTEE UNDER OUR NATIONAL SCIENCE AND TECHNOLOGY COUNCIL, SO THAT GROUP IS CHARTERED THROUGH NEXT YEAR AND IT'S ALL OF THE SENIOR DATA LEADERS THAT HAVE RESPONSIBILITIES ACROSS THE ENTERPRISE OF THEIR ORGANIZATION ORGANIZATION, SO ED KEARNS IS ONE OF THE CDO'S AS A PART OF THAT GROUP, IN ADDITION TO ADDITIONAL DEPARTMENT LEADERS. AND THAT GROUP IS RESPONSIBLE FOR MAKING SURE THAT DATA INFLUENCES THE NATIONAL SCIENCE AND TECHNOLOGY INVESTMENT THAT WE MAKE ACROSS THE ADMINISTRATION. SO WORKING CLOSELY WITH THE NIDR FOLKS, INFORMING THE DECISIONS MADE ACROSS THE ADMINISTRATION, THEY'RE ACTUALLY WORKING RIGHT NOW TO PUBLISH A PERIL DATA CHALLENGE -- A FEDERAL DATA CHALLENGE REPORT, SHOULD BE OUT IN DECEMBER, IT WILL BE A PRIOR TIERPZED SET OF DATA CHALLENGES THE CDO'S HAVE IDENTIFIED AND THAT WILL BE PUBLISHED TO THE COMMUNITY IN ORDER TO INFORM THE INVESTMENT DECISIONS NOT ONLY WITHIN THE AGENCIES BUT PRIVATE SECTOR AND ACADEMIA AS WELL. SO SETTING A DATA AGENDA. THAT SHOULD BE OUT IN DECEMBER. WE'LL HAVE A HUGE EVENT, YOU KNOW HOW THE WHITE HOUSE DOES. AND WE'LL USE THAT AS AN OPPORTUNITY TO ENGAGE WITH THE PRIVATE SECTOR AND ACADEMIA TO TALK ABOUT SOLUTIONS TO THESE CHALLENGES. SO AS WE PRODUCE THAT REPORT, ON THINGS THAT COULD BE DONE IN THE DAYS OF THE THOSE CHALLENGES. OH, AND THEN OPPORTUNITY PRACTICES. SO ONE OF THE OTHER THINGS WE'VE DONE IN ADDITION TO STANDING UP THESE IDENTIFIED SOME -- IN PARTICULAR TO HELP AGENCIES IDENTIFY MECHANISMS FOR GETTING THEIR DATA USED BY THE TECH COMMUNITY, SO THE OPPORTUNITY, WE DID AN AGENCY COHORT ABOUT -- WEEKS AGO CREATE USE CASES , AND USE THE USE CASES AND WITH THE TOOL. FAST-FORWARD FOUR OR SIX WEEKS, WE HAD ABOUT 20-PLUS TOOLS DEVELOPED, IT WAS A HUGE SUCCESS SUCCESS, WE WANT TO SEE WORKING CLOSELY WITH JUSTIN AND AUSTIN AND NOW BURTON, WE'LL WORK IT OUT SO THAT COMMERCE COULD TAKE THE LEAD IN MAKING SURE THAT THIS PROCESS IS AVAILABLE TO BE A TO HAVE A MECHANISM FOR ENGAGING IN WORKING TECH SECTOR. SO ALL THE AGENCIES ARE EXCITED, AND WE ALREADY HAVE A NEW SET OF USE CASES THAT WE'LL BE WORKING WITH COMMERCE AND OTHERS TO GO FORWARD WITH. SPEAKER: NAT LIRKS I KNOW WE'RE SHORT ON TIME TIME, BUT I WANT TO GIVE AN OPPORTUNITY, I WAS ALSO STRUCK WHEN I'VE GONE TO THE DATA CABINET, HOW MANY COMMERCE LEADERS ARE REALLY INVOLVED, EVERYBODY FROM JEFF TO OZZIE, BE BENDER, NATASSJA, TOM INCREDIBLE KEARNS, TOM BEACH, TO HAVE THAT KIND OF ASSISTANCE. I DON'T KNOW IF YOU WANT TO COMMENT ON THAT OR IF ANYBODY HAS ANY QUESTIONS FOR NATALIE. NATALIE: NO? GREAT. THANK YOU. SPEAKER SPEAKER: THANK YOU. SPEAKER: I'LL BE MAKING SURE THAT WE GET IT ALL OUT. CONTINUE THE DIALOGUE. THANKS, NATALIE. SPEAKER: THANKS SO MUCH. OKAY. YOU ALWAYS LIKE TO BE THE CROWN CROWNING PRESENTATION RIGHT BEFORE LUNCH, NATASSJA. (LAUGHTER). SO WE HAVE ONE OF THE THINGS WE'RE VERY PROUD ABOUT IS THE COMMERCE DATA ACADEMY, AND I JUST WANT TO PULL IT UP, IT ONE ONE OF THE AWARDS -- IT WON ONE OF THE AWARDS FROM, WHAT WAS IT THE -- ARE SPEAKER: FED SCOOP. SPEAKER: FED SCOOP, PROGRAM OF THE YEAR. NATASSJA HAS BEEN LEADING THAT EFFORT AND BURTON WILL BE PARTICIPATING IN THIS TOO, WHO HAS BEEN DOING A LOT OF WORK ON THE ACADEMY AND OUR DISCUSSIONS FOR THIS ARE KATY AND HEATHER. SO GO AHEAD. SPEAKER: GREAT. I FEEL LIKE A LOT HAS ALREADY BEEN SAID ABOUT THE COMMERCE DATA, ACTUALLY HAVING SOME OF OUR GRADUATES PRESENT GIVES YOU, THAT'S THE BEST ADVERTISEMENT WE COULD ASK FOR, TO HAVE STEVEN DEVINE AND ANDREA HIGHLIGHT IT. JUST A LITTLE BACKGROUND ON WHERE WE CAME FROM . SO DATA EDUCATION IS ONE OF THE THREE SERVICES THAT WE OFFER, AND THAT'S SOMETHING I'M COMPLETELY FOCUSED ON IN MY CURRENT ROLE. THE PRINCIPLES OF HOW WE WORK WITH PRODUCTS AND SERVICES, WE ALSO APPLY TO COMMERCE DATA ACADEMY, SO COLLABORATIVE, AGILE, LEAN, USER USER-CENTERED DESIGN, DRIVEN BY OUTCOMES AND OPEN BY DEFAULT, AND AS KATY MENTIONED EARLIER, WE ARE MAKING ALL OF OUR STAFF- STAFF-TAUGHT COURSES AVAILABLE ONLINE FOR FREE WITH THE MATERIALS THEY USE TO ANYONE WHO IS INTERESTED IN WATCHING THOSE VIDEOS. SO THE THREE AREAS THAT DATA SCIENCE AND SOFTWARE ENGINEERING. WHEN WE FIRST STARTED, WE JUST DID A PILOT OF FOUR COURSES AND WERE ABSOLUTELY RESPONSE WE GOT. WE HAD OVER 300 PEOPLE IF FOR THESE COURSES AND REALIZEED VERY QUICKLY WE NEEDED TO FORMALIZE THE PROGRAM SO THAT WE COULD, I DON'T KNOW, CONTAIN THE GROWTH AND DIRECT IT WERE OUR SHORT COURSES THAT WE STARTED OUT WITH, AND THEN IN FEBRUARY OF THIS YEAR WE DID THE IMMERSIVE BOOTCAMP AND THOSE WERE THE GRADUATES THAT I'VE SPOKEN TO EARLIER. SO RIGHT NOW WE HAVE THREE DIFFERENT PROGRAMS WE OFFER. FIRST IS OUR SHORT COURSES, WHICH ARE GENERALLY THREE HOURS IN LENGTH AND WE HAVE ABOUT 30 PEOPLE IN PERSON AND ABOUT 250 ONLINE VIA WEBEX FOR LIVE STREAMING. SO LIVE COURSES ARE A VERY IMPORTANT PART OF OUR PROGRAM AND THE STUDENTS CAN ASK QUESTIONS, AND THEN OUR IN RESIDENCE PROGRAM, WE HAD 13 PEOPLE GRADUATE FROM THE PROGRAM AND THEY FINISHED WITH US IN SEPTEMBER OF THIS YEAR, SO THEY'RE JUST FINISHING UP AND GOING BACK TO THEIR BUREAUS NOW WITH THESE BEAUTIFUL OF THE TRAINING THAT WE OFFER IS TRAINING FOR LEADERSHIP, AND WE JUST LAUNCHED THAT LAST WEEK, REALLY EXCITED ABOUT THE RESPONSE. IT WAS TARGETED SPECIFICALLY TO OUR SPONSORING BUREAUS AND THEIR UPPER MANAGEMENT IN TRAINING THEM IN PRINCIPLES OF DATA SCIENCE, IN USE CASES AND BUILDING TEAMS AND TOOLS AND METHODS AND ALSO DATA PRODUCT, SO WE'VE HAD GOOD RESPONSE ON THAT SO FAR TOO. WORKING WITH US OR EVEN AFTER THE COURSE, ALL SORTS OF ONLINE RESOURCES THAT THEY COULD CONTINUE TO USE TO LEARN AND DEVELOP SKILLS. THESE ARE SOME OF OUR STATISTICS CHVMENT WE HAD ANOTHER COURSE YESTERDAY AND I DESCRINT A DHOONS PUT THOSE ON THE SLIDE, BUT WE ARE CONTINUALLY GROWING THE NUMBER OF PEOPLE THAT ARE TAKING OUR COURSES AND WE HAVE A LOT OF PEOPLE THAT ARE COMING BACK FOR MORE. SO JUST A COUPLE OF TESTIMONIALS FROM OUR SHORT COURSES. WE HAVE MOSTLY BEEN DOING BEGINNING LEVEL COURSES, BUT WE'RE HOPING TO BUILD IT OUT ADVANCED COURSES, AND I THINK THE IN RESIDENCE PROGRAM IS A WAY TO REALLY DEVELOP THOSE SKILLS IN DEPTH FOR PEOPLE THAT ARE PARTICULARLY INTERESTED AND MOTIVE AID -- AND MOTIVATED. SPEAKER: THAT'S TERRIFIC. SPEAKER: REAL QUICK ON THE INVESTMENTS PROGRAM, WE ARE REALLY MAKING SURE THAT WE MODULATE THAT EFFECTIVELY GOING FORWARD SO WE DON'T OVERTAX THE EXISTING STAFF IN THE CDS AND THAT WE LINE UP THE MENTORS EFFECTIVELY AND AS WE BUILD THE STAFF, WE'RE THINKING ABOUT THE BEST WAY TO LAUNCH THE NEXT ROUND OF THE INVESTMENTS PROGRAM PROGRAM, WHICH WON'T HAPPEN UNTIL AFTER THE NEW YEAR, BUT AGAIN, THAT'S ALSO VERY DRAINING FOR THE STAFF LAST TIME, SO WE WANT TO MAKE SURE THAT WE MODULATE THAT. SO WE SHOULD MOVE TO DISCUSSION AFTER YOU WRAP UP QUICKLY. SPEAKER: WE HAVE A WEBSITE, DATA ACADEMY. ACADEMY.COMMERCE.GOV, SO IT'S VERY APPROPRIATE WE'RE MEETING IN THIS ROOM, THIS IS WHERE WE HOLD OUR COURSES, SO YOU GET TO EXPERIENCE WHAT IT'S LIKE TO BE A STUDENT IN THE COMMERCE DATA ACADEMY. (LAUGHTER). AND CONGRATULATIONS ON OUR AWARD AWARD, TECH PROGRAM OF THE YEAR. (APPLAUSE). ARE SPEAKER: THANK YOU. WILL RR ARE SPEAKER: AMAZING. OBVIOUSLY THIS IS ONE OF THE CROWNING JEWELS. I WANT TO FOLLOW UP ON SOMETHING BURTON THAWTIOND. WHY DOES THE IN RESIDENCE PROGRAM, WHY DO YOU FEEL THAT IT TAXED THE INSTRUCTORS? SPEAKER: WE ONLY HAD A STAFF OF TEN PEOPLE AND WE HAD 15 RESIDENTS, AND SO OUR STAFF WERE EXPECTED TO WORK ON THEIR OWN PROJECTS AND HELP THESE 15 PEOPLE WHO WERE BRAND-NEW AT CODING GET UP TO SPEED AND GET GOING ON THEIR PROJECTS, SO IT WAS A SORT OF DUAL -- SPEAKER: WEAPON DIDN'T ROIRPZ THE FIRST TIME AROUND -- WE DIDN'T RECOGNIZE THE FIRST TIME AROUND HOW MUCH MENTORSHIP WAS NEEDED AND HELP FOR THE DETAILEES COMING IN, SO NOW WE WANT TO STRUCTURE THAT MENTOR MENTORSHIP SO THAT IT'S WORKED IN MORE EFFECTIVELY TO THE EXPECTATIONS OF THE STAFF ON THE CDS TEAM SO THAT THEY'RE NOT JUST ALL OF A SUDDEN WORKING 40 HOURS A WEEK AND ALL OF A SUDDEN ASKED TO DO 10 MORE HOURS OR WHATEVER WORKING 60 AND ASKED TO FIGURE OUT. SPEAKER: YOU'RE BALANCE LG WORKLOAD EXPECTATIONS SPEAKER: RIGHT. WE'LL ALSO TWEAK THE IMMERSIVE BOOTCAMP BECAUSE WE FELT LIKE IT DIDN'T PREPARE THEM ADEQUATELY ENOUGH FOR THE WORK THAT THEY NEEDED TO DO. SO THAT'S ANOTHER THING WE CAN DO TO ENHANCE THE EXPERIENCE EXPERIENCE. SPEAKER: I WANT TO MAKE TWO QUICK COMMENTS. ONE IS THAT I LOVE THE FACT THAT WE'RE IN A RESEARCH LIBRARY, ALL THE WORK THAT WE'RE DOING IN TERMS OF TRYING TO TRAIN 21ST CENTURY LIBRARIANS IS REALLY AROUND DATA MANAGEMENT AND THE CONCEPT OF AN ACADEMY FOR THIS KIND OF TRAINING IS SOMETHING THAT THE RESEARCH LIBRARY COMMUNITY IS DEEPLY INTERESTED IN, SO I HOPE WE'LL BE ABLE TO BUILD SOME CONNECTIONS BETWEEN THESE TWO COMMUNITIES, ESPECIALLY GIVEN THE FANTASTIC WORK YOU'VE DONE IN THIS PROJECT. WHEN SECRETARY PRITZKER SPOKE THIS MORNING, SHE TALKED ABOUT THE NEED TO MARKET AND BRAND THE OUTPUTS OF THE WORK THAT YOU ALL ARE DOING, AND IT STRUCK ME THAT JUST AS YOU SAID, THAT YOU COULDN'T HAVE A BETTER ADVERTISEMENT FOR THE SUCCESS OF THE ACADEMY THAN SOME HERE. I WAS THINKING LITTLE VIDEOS OF THEM ON YOUR WEBSITE WOULD GO SUCH A LONG WAY IN BUILDING THE UNDERSTANDING OF JUST HOW INCREDIBLY EFFECTIVE THE PROGRAM THAT YOU'RE BUILDING THINKING ABOUT FEATUREING THESE GUYS IN LIVING COLOR. SPEAKER: PLANS TO RELEASE FOR EACH ONE AND ANNOUNCEMENTS. SPEAK ARE: THANK YOU. I KNOW THE RESOURCES AND WHERE THEY'RE AVAILABLE, SO I'LL STOP HERE. (LAUGHTER). SPEAKER: I REALLY LIKE THE IDEA OPEN SITES SUCH AS -- ARE BOEING HAS A MASSIVE SET OF 1200, AND STILL CAN'T TRAIN FAST ENOUGH BECAUSE TECHNOLOGY IS PROGRESSING SO QUICKLY AND SO VIGOROUSLY. SO JUST KEEPING YOUR OWN INTERNAL PEOPLE TOWP DATE WITH ALL THE TECHNICAL DEVELOPMENTS IS HARD, EVEN FOR MAJOR COMPANIES. SO I THINK YOU WILL ALSO GET TO THE POINT THAT THEY'LL BE PARTNERING MORE AND MORE WITH ACADEMIC INSTITUTIONS BUT ALSO POTENTIALLY THESE COMPANIES TO GET REALLY ALL THE COURSES YOU NEED BECAUSE CREATING THIS IN-HOUSE IS HARD . AM TRAINING TRAINERS OR TEACHING TEACHERS IS WHAT YOU'RE DOING IS TO TRAIN THE FIRST GENERATION, THAT THEN TRAINS THE SECOND GENERATION IN THEIR OWN DEPARTMENTS OR WITHIN THE BUREAU, AND I ALSO REALLY LIKE THE IDEA TO FEATURE THOSE WHO DID THAT EXTRA EFFORT AND HOW MUCH FASTER AND BETTER THEY CAN DO THEIR JOB AND POTENTIALLY ALSO HOW MUCH MORE SATISFIED THEY ARE WITH WHAT THEY CAN DO NOW. SPEAKER: THAT'S A TERRIFIC POINT. SPEAKER: THE QUESTION ON HOW THIS WOULD INTERACT WITH RESOURCES THAT EXIST ON THE NET FOR TRAINING, BUT THERE ARE A LOT OF TRAINING COURSES OUT THERE FOR SOME OF THE THINGS YOU'RE DOING, BUT SOME OF THE TRAINING YOU'RE DOING UNDOUBTEDLY IS UNIQUE, AND HAVE YOU CONSIDERED TAKING YOUR COURSES AND PUTTING THEM ON EDEX JEWEL FOR DATA -- FOR THE DATA DATA -- THE DATASETS AND SO FORTH. BEING SORT OF BROADLY RECOGNIZED AS A TEACHER IN THIS AREA WILL GO A LONG WAY TOWARDS GETTING PEOPLE INVOLVED IN USING THE DATA AFTER THEY GET IT OUT. SO ON EDEX, ONE OF THE TOP THREE TO FIVE COURSES IN THE WORLD OF DATA SCIENCE COMES FROM THE COMMERCE DEPARTMENT. SPEAKER: YES, AND WE'RE ACTUALLY WORKING ON ALL OF THE COURSES THAT WE'VE RECORDED WE'RE CLOSED CAPPINGING RIGHT NOW SO THEY'LL BE 508 COMPLIANT AND THEN WE WOULD DEFINITELY LIKE TO MOVE FORWARD, EVEN SOME WHAT KATY TALKED ABOUT ABOUT, THE MASSIVE OPEN SOURCE ONLINE COURSES, WE WANT TO GET INTO THAT COMMUNITY AS WELL, AND OUR STAFF, THESE SAME STAFF THAT WE'RE EXPECTED TO MENTOR AND DO THEIR OWN JOBS ARE ALSO TEACHING GIVE THEM VISIBILITY AND HAVE THEM SEE THE BENEFIT OF THAT PROFESSIONALLY AS WELL. SO I MEAN, IT REALLY SERVES A LOT OF DIFFERENT PURPOSES ALL FOR GOOD. THANKS. SPEAKER: I WANT TO ASK WHAT TYPE OF ALUMNI TYPE PROGRAMS YOU'RE THINKING ABOUT FOR YOUR GRADUATES AND IF YOU'VE THOUGHT, YOU KNOW, JUST KIND OF EVEN THE SHORT-TERM AND LONG-TERM, IF YOU WANT TO PUT IN PLACE THERE, BECAUSE THAT'S OFTEN VERY IMPORTANT IN DEVELOPING BOTH KIND OF IMPACTED AND CULTURE AND KEEPING THE NETWORKS ALIVE BUT ALSO JUST KEEPING PEOPLE REMEMBERING, YEAH YEAH, THEY STARTED THERE, EVEN IF THEY GO SOMEWHERE ELSE AND KEEPING THOSE KIND OF CONNECTIONS ALIVE. SPEAKER: RIGHT. WE ONLY HAVE A FEW ALUMNI YET, BUT WE HAVE A CHANNEL FOR THEM SPECIFICALLY, AND WE'RE ALSO -- I DON'T KNOW, WE'RE OPEN TO SUGGESTIONS ON THAT. THAT'S A NEW THING BECAUSE WE ONLY JUST GRADUATED THEM IN SEPTEMBER. BUT I THINK THE KEY THING IS GIVING THEM OPPORTUNITIES AND GIVING THEM SUPPORT IN THEIR BUREAUS TO CONTINUE DEVELOPING THEIR SKILLS BECAUSE THESE SKILLS, IF YOU DON'T USE THEM, YOU LOSE THEM VERY QUICKLY. SO THAT WOULD BE ONE THING WE WOULD BE LOOKING TO SUPPORT ONGOING. THEY'RE NOT GOING TO BE ABLE TO CONTINUE AT THE PACE THEY'VE BEEN GOING BECAUSE THEY'RE DOING THEIR OTHER JOBS NOW, BUT I THINK IT'S A GOOD THING TO START A CONVERSATION AROUND. I DON'T KNOW, WHAT OTHER KINDS OF A AALUMINUM ANY NETWORKS COULD WE WE -- DOES ANYONE ELSE HAVE ANY THOUGHTS ON THAT? SPEAKER: VERY GOOD POINT. WE'RE THINKING ABOUT THE ALUMNI NETWORK FOR THE DETAILEES BUT ALSO FOR THE COHORT OF FOLKS WHO ACTUALLY COME THROUGH CDS, RIGHT? SO THE AND WE'RE IMPLEMENTING THIS. WE'RE GOING TO HAVE SOME MEMBERS ARE GOING TO BE LONG-TERM EMPLOYEES, AND WE'LL HAVE FOLKS THAT COMMIT FOR ONE YEAR AND THREE YEARS, AND WE'RE HAPPY WITH THAT THAT, RIGHT? WE'RE HAPPY WITH THE IDEA THAT WE'RE GOING TO HAVE COHORTS, AND I KNOW IT'S ON SPECIAL JEFF'S RADAR TO THINK ABOUT THAT, JEFF CHEN AND JEFF M MEISL HAVE BEEN THINKING ABOUT THAT, HOW DO WE CAPTURE THE SPIRIT OF THE COHORTS AND TO KEEP THAT NETWORK GOING AS WE GO HE SPEAKER: ARE YOU DOING MICROCREDENTIALS FOR THIS OR MICRODEGREES FOR THIS? SPEAKER: OH, LIKE CERTIFICATES? SPEAKER: YEAH. SPEAKER: SPECIFICALLY SOMETHING LIKE THAT SO THEY HAVE -- SPEAKER: SPECIFICALLY THINGS LIKE THAT THAT CAN BE PICKED UP AND TRANSFERRED TO OTHER DEPARTMENTS DEPARTMENTS WITHIN THE GOVERNMENT. IT WOULD BE GREAT IF THE OUTCOME OF THIS WOULD STICK WITH THE EMPLOYEE WHO TOOK AROUND. SPEAKER: OH, I SEE. YEAH, LIKE THOSE LITTLE -- SPEAKER: LIKE PROGRAM MANAGERS, GREAT SUGGESTION, WE'LL LOOK INTO THAT. SPEAKER: DEFINITELY DEFINITELY. SPEAKER: OKAY. WE'RE ONLY A FEW MINUTES LATE, WHICH IS COOL. WE HAVE A WORKING LUNCH SET UP WITH THE CDAC AND MEBLGZ OF OUR EXECUTIVE DATA EXECUTIVE COUNCIL. WE'LL BE GOING TO THE BACK OF THE ROOM ROOM. THAT'S THE NEXT STEP. SO WE'RE ADJOURNING FOR LUNCH AND WE'LL RECONVENE HERE AT ABOUT 12:20. WHAT'S THAT? 1:20. 12:20 WOULD BE A LITTLE FAST. (LAUGHTER). YEAH. SO LET'S GATHER IN THE BACK AND PEOPLE WILL ESCORT US. SPEAKER: ALL RIGHT. THANK YOU. YES, YOU CAN LEAVE STUFF HERE. ( H BREAK >> BREAK UNTIL 1:20 P.M. EST. (ON BREAK UNTIL 1:20) TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST >> WE'LL GET STARTED IN ABOUT ONE MINUTE. LET'S COME BACK TOGETHER AND GET STARTED AGAIN. I'LL GET JUSTIN TAKE IT FROM HERE. WE MOVE INTO A DISCUSSION WITH DANAH BOYD. >> ALL RIGHT. WHY DON'T WE HAVE EVERYBODY RECONVENE HERE AND I ALWAYS KNOW THAT RIGHT AFTER LUNCH PEOPLE CAN COME IN A LITTLE SLEEPY EYED SO WE WANTED TO WAKE EVERYBODY UP AND WE THOUGHT ONE OF THE BEST WAYS TO DO THAT WOULD HAVE DANAH BOYD LEAD OUR NEXT SESSION. EXACTLY. I HAVE -- OBVIOUSLY REALLY BENEFITED FROM A NUMBER OF MEET MEETINGS AND DISCUSSIONS WITH DANAH, AS WE HAVE BEEN THINKING ABOUT THOUGHT LEADERSHIP ON THIS AREA. THE LAST COUPLE OF O MONTHS WE HAVE BEEN SPEAK EGG ABOUT THE IDEA IN A QUALITY HOW DO YOU MAKE SURE THE DATA WE HAVE IS GETTING TO ALL PARTS OF SOCIETY SO THEY CAN USE IT, WHO SHOULD BE USING IT, OUR CHARITIES, NON-PROFITS SMALL AND MEDIUM SIZE ENTITIES ENABLED TO USE AND DRIVE GROWTH OR SOLUTIONS THROUGH OUR DATA OR CANDIDLY BY OPENING UP OUR DATA AND BUILDING THESE TOOLS IS THE DATA THEN GOING TO THE SAME CONSUMERS THAT ARE ALREADY USING IT. IT'S KIND OF AN ANALOG THE WAY WE HAVE BEEN TALKING ABOUT INCOME INEQUALITY, THE INCOME INEQUALITY AS A SOCIAL AND ECONOMIC ISSUE IS SOMETHING HERE AT COMMERCE WE HAVE BEEN VERY WORRIED ABOUT, THE PRESENCE BEEN VERY WORRIED ABOUT AND WE SAW POTENTIAL FOR THE SAME CONCERNS ABOUT DATA, IF YOU THINK ABOUT IT AS AN ASSET. ARE WE REALLY MAKING SURE THAT WE'RE SETTING IT UP SO THAT EVERYBODY CAN USE IT, BENEFIT, GROW, ADDRESS PUBLIC PROBLEMS. DANAH IS OBVIOUSLY A THOUGHT LEADER ON THIS ISSUE SO I WILL TURN IT OVER FOR WHAT I'M SURE WILL BE AN INTERESTING DISCUSSION. >> HE PUTS IT IN NICE POLITICAL TERMS, HE BASICALLY TOLD ME TO BE PROVOCATIVE. WHICH IS FINE. AND MY TALK TODAY WILL TALK ABOUT CHALLENGES WE FACE AND IN RETURN CHALLENGES AND BIG ASSUMPTIONS THE THINGS WE HOLD DEAR BECAUSE MANY HOLD THE BELIEF TECHNOLOGY IS BEING AMAZING OPPORTUNITY AND I DO TOO. BY GIVING EXAMPLES WAYS OF CHALLENGING OR THINKING. WE HERE BECAUSE WE SEE POTENTIAL. WE IMAGINE THE POTENTIAL. WE DON'T NECESSARILY THINK ABOUT ADVERSARIES, WE THINK ALL OPPORTUNITIES WE CAN USE TECHNOLOGY TO MAKE SOCIETY BETTER. ONE OF OUR PASSION PROJECT COMMITMENT TO THINKING ABOUT DATA, START TO HURT THE PEOPLE WE'RE TRYING TO HELP? WHAT IF WE'RE BUILDING A PROJECT TO INCREASE INEQUALITY? THAT'S THE FUNDAMENTAL QUESTION I WANT TO ASK AND I WANT TO PLAY WITH. SO HOW MANY OF YOU THINK DISCRIMINATION IS A BAD IN THIS CASE? THE SILENT STIGMA TO GUILT YOU, BUT IN SOME WAYS I GAVE YOU A COMPLETE TRICK QUESTION. BECAUSE ALL DEPENDS ON HOW WE DEFINE DISCRIMINATION. WHEN THINK OF IT MOST THINK ABOUT THE UNJUST PREJUDICIAL TREATMENT BASED ON PROTECTIVE CATEGORIES THAT WE OFTEN TALK ABOUT IN TERMS OF SOCIAL ISSUES. DISCRIMINATION HAS A MATHEMATICAL AND ECONOMIC ROOT, IT'S CORE TO WHAT WE DO WHEN WE DO DATA SCIENCE. THE PRACTICES OF DATA CLEANING, CLUSTERING DATA, RUNNING STATISTICAL CORRELATIONS, ET CETERA, ARE ALL PRACTICES THAT ARE ABOUT USING INFORMATION TO DISCERN ONE SET OF INFORMATION FROM ANOTHER. THE BIG QUESTION PRESENTED BY DATA PRACTICES IS WHO GETS TO CHOOSE WHAT ARE THE ACCEPTABLE BOUNDARIES WE OPERATE WITH. WHO GETS TO DECIDE THE VALUES AND TRADE OFFS OF A GIVEN PRIORITY? WHO GETS TO MAKE THE DECISIONS THAT DO ANALYSIS THAT SEGMENT DATA AND WHEN WE SEGMENT DATA WE END UP SEGMENTING PEOPLE IN INTERESTING WAYS. I WOULD ARGUE THERE IS NOTHING ABOUT DOING DATA ANALYSIS THAT IS AT ALL NEUTRAL. WHAT HOW IT'S COLLECTED HOW'S CLEANED AND STORED, WHAT MODEL ARE CONSTRUCTED AND WHAT QUESTIONS ASKED, ALL THIS IS DEEPLY POLITICAL PROJECT. MORE OFTEN THAN NOT WHEN WE TALK TECHNICAL TERMS WE LIKE TO PRETEND THE POLITICAL ASPECT DID NOT PLAY OUT, THAT WE CAN GET AWAY FROM THE POLITICAL, WE DON'T HAVE THINK ABOUT THE POLITICAL. BUT I DON'T WANT TO PUSH BACK AND SAY WE CANNOT PUNT THIS DOWN LINE, WE HAVE TO ACKNOWLEDGE POLITICAL AT EVERY TURN IF WE'RE GOING TO DO DATA SCIENCE AND PROJECTS RIGHT. WE HAVE TO ACKNOWLEDGE WHERE TURN IN UNACCEPTED WAYS AN ACCOUNT FOR IT IN OUR DESIGN, OUR DEVELOPMENT AND OUR DEPLOYMENT. SO I'M GOING TO GIVE A COUPLE OF EXAMPLES WHERE THIS PLAYS OUT IN DIFFERENT PARTS OF PROCESS. AND I'LL BEGIN BY ACTUALLY TALKING ABOUT WHERE OPEN DATA CAN PLAY OUT INTO A PROJECT WE DIDN'T NECESSARILY HOPE FOR. THIS CONCERNS WHAT IS THE RULE OF OPEN DATA? MOST OF US REALLY HAVE A OPEN DATA GOOD BELIEF. THAT BELIEF COMES FROM A BELIEF IN INFORMING THE PUBLIC, GETTING PEOPLE ENGAGED -- THE KNOWLEDGE WILL BE EMPOWERING, THAT KNOWLEDGE WILL LEAD US TO -- BUT WHAT IF THAT HAPPEN? IN NEW YORK CITY, THERE IS DEPARTMENT OF EDUCATION, OPEN UP DATA ABOUT SCHOOLS. THROUGH SCHOOL PERFORMANCE DASHBOARD, SOME OF YOU MAY BE FAMILIAR WITH DASHBOARD LIKE. THIS IF YOU DENT LIKE THE INTERFACE YOU CAN USE OTHER SERVICE LIKE SCHOOLS ALSO POWEREDDED BY THE NEW YORK CITY DEPARTMENT OF EDUCATION DATA. THE PURPOSE OF THE SYSTEMS IS WHAT WE IMAGINE WHEN WE BUILD AN OPEN DATA PROJECT. LET'S EMPOWER PEOPLE, PARENTS AND FAMILIES TO MAKE -- AROUND SCHOOL FOR THEM CHILDREN AND FAMILY AND COMMUNITY. BUT SCHOOL CHOICE IS AN EXTREMELY POLITICAL MATTER. AND THE VALUE DATA DEPARTMENT OF EDUCATION COLLECTS RUNS STRAIGHT INTO THE FRONT NATURE OF THE VALUE OF EDUCATION. WHAT MAKES A SCHOOL GOOD? IS IT ABOUT TEST SCORES? STUDENT MAKE UP, PARENT RATINGS? DIFFERENT FAMILIES HAVE DIFFERENT VALUES SO THEY READ THE DATA IN DIFFERENT WAYS. AND THIS IS OFTEN CONSIDERED A FEATURE BECAUSE WE WANT PEOPLE TO CHOOSE THE SCHOOL THAT'S BEST FOR THEM. AND WE TAKE IT FROM THIS VERY INDIVIDUAL CENTRIC PERSPECTIVE THAT IF WE ACTUALLY EMPOWER EACH FAMILY TO HAVE ALL THE INFORMATION THAT THEY CAN POSSIBLY GET THEIR HAND ON, THEY WILL MAKE AN INFORMED MEANINGFUL CHOICE AND IT WILL BE GOOD. SCHOOL CHOICE BASED ON DATA PRESENCE A SERIES OF CHALLENGES. FIRST ARE CHALLENGES WE NORMALLY THINK ABOUT. THIS IS THE FACT THAT WE DEAL WITH POPULATION WHOSE DON'T NECESSARILY KNOW HOW TO READ STATISTICS. THE GENERAL POPULATION DOESN'T KNOW HOW TO READ STATISTICS. AND NEW YORK CITY WE'RE DEALING WITH FAMILIES WHO DON'T KNOW HOW TO READ ENGLISH. SO THIS IS WHERE WE SEE THIS MOMENT WE'RE NOT NECESSARILY COMETOLOGY THE TABLE EVENLY -- COMING TO THE TABLE EVENLY. SECOND SCHOOL RANKING THE ACTION YOU CAN TAKE WHEN YOU HAVE SCHOOL CHOICE, IS ACTUALLY DEPENDENT ON GEOGRAPHY. SO MAKING INFORMATION AVAILABLE ABOUT ALL THESE DIFFERENT COMMUNITIES YOU CAN FIND OUT THAT THERE ARE PHENOMENAL SCHOOLS IN MANHATTAN AND IF YOU LIVE IN QUEENS THERE'S NOTHING YOU CAN DO YOUR CHOICES ARE CONSTRAINED BY OTHER DECISIONS SO THE FACT YOU HAVE THIS INFORMATION MAKES YOU FEEL MORE DISCONNECTED FROM THE ABILITY TO DO SOMETHING, THE PEOPLE WHO HAVE ECONOMIC POWER CAN GET UP AND MOVE AND GUESS WHAT, FAMILIES DO THAT ALL THE TIME IN NEW YORK. SOME FAMILY VERSUS MORE TIME TO DEVOTE TO MAKE SENSE OF THE THAT DASHBOARD, WHAT IT MEANS TO THINK THROUGH WHAT IT MEANS FOR THEM AND IF THEY DON'T HAVE TIME THEY CAN OUTSOURCE IT, THERE'S TONS OF ORGANIZATIONS IN NEW YORK CITY THAT YOU CAN ACTUALLY PAY TO WALK THROUGH ALL THE DATA FOR YOU TO SURVEY YOU AND FIGURE OUT WHAT THE RIGHT SCHOOL CHOICE IS. AGAIN, ECONOMIC DIVISION THAT'S HAPPENING. YOU GET THE IDEA. THESE ARE THE KINDS OF CLASSIC UNEVENNESSES WE'RE REMINDED THAT OPENNESS DOESN'T NECESSARILY ACHIEVE INEQUALITY GAP. BUT THAT'S ACTUALLY NOT EVEN THE BIGGEST PROBLEM FOR THIS OPEN DATA PROJECT. IT'S NOT DISCUSS IN ALL THIS PROCESS, IS HOW PUBLIC GOOD AND INDIVIDUAL DESIRE ARE OFTEN IN CONFLICT AND NOWHERE IS THAT MORE CRITICAL THAN WHEN WE THINK ABOUT THE PRODUCT OF EDUCATION. WHAT IS THE PROJECT OF EDUCATION? ARE WE WORKING ON SKILL DEVELOPMENT OF INDIVIDUALS OR ARE THERE MORE COMMUNITY PROJECT COMPONENT TO IT ALL? (INDISCERNIBLE) IS A RESEARCHER AT HARVARD WHO HAS DONE FANTASTIC WORK LOOKING AT HOW HARDY VERSETY IS IN THE WORK FORCE. MANY OF YOU KNOW THESE STORY THE DIVERSE WORK FORCE IS THE MORE SUCCESSFUL IT IS AND A WHOLE SET OF MEASURES BUT WHAT HE'S SHOWN CONSISTENTLY IS THE MORE DIVERSE A WORK FORCE IS THE LESS HAPPY PEOPLE THINK THEY ARE. AND THE LOWER THEY THINK THEIR PERFORMANCE IS. THEIR PERCEPTION OF PERFORMANCE IS NOT ACTUALLY CONNECTED TO ACTUAL MEASURE MEASURES. THEY'RE CONNECTED TO FEELING, THE EMOTIONAL ASPECTS OF IT. GUESS WHAT? SCHOOL CHOICE PROMPTS THE SAME EMOTIONAL REACTIONS. WE SEE THE SELF-SEGREGATION THAT WE WOULD SEE IN ANY OTHER WORK FORCE UNLESS YOU MAKE A COMMITTED EFFORT TO ACTUALLY TRY TO GET PEOPLE TO ENGAGE ACROSS DIFFERENCES. IN NEW YORK CITY BLACK FAMILIES CHOOSE TO SCHOOLS THAT ARE PREDOMINANTLY BLACK, WHITE FAMILIES CHOOSE SCHOOLS PREDOMINANTLY WHITE AND NEW YORK CITY HAS THE HIGHEST RATE OF SELF-SEGREGATION OF ANY CITY IN THE COUNTRY. THE MORE WE OPEN DATA THE MORE WE SELF-SEGREGATED. 'S AN INTERESTING CHALLENGE BECAUSE IT'S ASSUMPTION THIS DATA WOULD BE GOOD BECAUSE WE WANT IT TO BE GOOD BUT WE'RE GRAMMING WITH THE REAL RAW EMOTION AND DYNAMICS AND WHAT IS THE QUESTION -- GRAPPLING WITH IT. WHAT'S THE QUESTION TO MAKE INDIVIDUAL CHOICES IN LIGHT OF VERY COLLECTIVE PROCESSES. SO WHEN WE OPEN UP DATA ARE WE EMPOWERING PEOPLE TO COME TOGETHER OR BECOME APART? ARE WE GIVING THEM THE TOOLS TO MAKE CHOICES FOR THEMSELVES AT THE EXPENSE OF THE COLLECTION AND WHO GETS TO DETERMINE THE VALUES AND THE TRADE OFF THAT ARE GOING ON THERE? IN SOME WAY WE'RE COMING TO A BASIC NOTION OF WHAT IS OUR DEMOCRACY? AND WHAT IS POPULISM AS RELATES TO DEMOCRACY? IT ISN'T CLEAR WHAT WE WANT OUT OF THESE PROJECTS. WHEN WE OPEN DATA AND DON'T KNOW THE CLEAR AGENDA, WHO IS MAKING THESE DECISIONS AND THAT'S WHERE THERE'S POWER VACUUMS THAT GET FILLED BY PEOPLE LOOKING TO RE REVERIFY EXISTING STRUCTURAL DYNAMICS AND SOME MAY NOT BE THE POLITICAL WORK WE WANT TO SEE DONE. SO LET'S TALK EQUITY. I INPUT EQUITY FRONT AND SENDER OF ANY EQUATION I CARE WHEN I THINK ABOUT DATA AND ACKNOWLEDGE THIS IS A DEEPLY POLITICAL COMMITMENT. IT'S SOCIAL GOOD MATTERS MORE THAN INDIVIDUAL QUALITY ISSUES AND IS SOMETHING THAT IS ACTUALLY CONTROVERSIAL WITHIN THIS CULTURAL ENVIRONMENT. AND PART OF THE PROBLEM IS DATA ACTUALLY FORCES US TO CONTEND WITH MOMENTS WHEN QUALITY AND EQUITY ARE IN CONTRADICTION. SO UNLESS YOU'RE A STATISTICIAN YOU PROBABLY HAVEN'T BEEN FOLLOWING THE DEBATES OVER NORTH POINTS COME PASS, A TOOL TO USE WHETHER OR NOT SOMEBODY ARRESTED DESERVES TO RECEIVE BAIL. MOST DON'T FOLLOW CRIMINAL JUSTICE IN DETAIL BUT BAIL IS SIGNIFICANT PROCESS IN THE U.S. SYSTEM BECAUSE IT IS DETERMINING A WHOLE SET OF THINGS AND OUTCOMES THAT WE CARE ABOUT. IF U YOU GET BAIL, YOU ARE MORE LIKELY TO KEEP YOUR JOB, HOUSE, CHILDREN, YOUR SPOUSE, ET CETERA. IF YOU DON'T GET BAIL YOU'RE MORE LIKELY TO PLEAD GUILTY, EVEN WHEN YOU'RE NOT. AND DUE TO LONGEVITY PEOPLE SIT IN JAIL, MANY PEOPLE PLEAD GUILTY, EVEN WHEN THEY ARE INNOCENT JUST TO GET BACK TO ACTUALLY HAVING A JOB. AND SO KEEP IN MIND 93% OF ALL CASES IN THE UNITED STATES ARE PLEAED OUT. HOW DOES A JUDGE FAIRLY DETERMINE IF SOMEONE DESERVES BAIL? THAT'S THE FRONT OF THE PROCESS, IF WE ASSUME, GUILTY, ASSUME, HOW DO WE DETERMINE WHETHER THEY'RE FREE AS THEY STAND TO WAIT FOR TRIAL OR WHETHER THEY'RE TO BE INCARCERATED IN THE IDEA WE WILL JAIL PEOPLE, WE WILL KEEP THEM INCARCERATED, WE WILL KEEP THEM FROM FREEDOM, IN ORDER TO PROTECT THE SAFETY OF THE PUBLIC. BUT WHO GETS TO MAKE THAT DECISION? HISTORICALLY WHAT WE KNOW IS JUDICIAL DECISIONS ARE EXTRAORDINARILY BIASED IF NOT FRANKLY RACIST. SO WHAT COMES INTO PLAY THE IDEA WE SHOULD USE MORE DATA TO FIGURE THIS OUT AND RISK ASSESSMENT TOOLS ARE DEVELOP AND DEPLOYED, BIG THING IN CRIMINAL JUSTICE, HOW TO EMPOWER JUDGES TO MAKE DECISIONS WHETHER OR NOT SOMEBODY IS A RISK? THESE TOOLS ARE DEVELOPED TO NEUTRALIZE THAT ANALYSIS TO HELP JUDGES MAKE BETTER DECISIONS. THEY DON'T DO THE WORKS JUDGE BUS INFORM JUDGES AND THEY ARE QUOTE UNQUOTE NEUTRAL THIRD PARTIES. THE REALITY IS THAT A JUDGE HAS GIVEN THAT INFORMATION AND THEY DON'T FALL FOR IT THEY'RE MORE TROUBLE THAN IF THEY DID ABIDE BIT AND ARE WRONG. A FEW MONTHS AGO CONTROVERSIAL ARTICLE BASED ON CHALLENGING BROWARD COUNTY PROCESS AND ARGUING IT WAS A RACIST PROCESS. INTEREST NOTES ONE OF MY FELLOWS WAS DEEPLY INVOLVED IN THAT WORK AND DID A LOT OF TECHNICAL ANALYSIS. THE (INAUDIBLE) CAME UP WITH -- SHOWED THAT BLACKS WHO NEVER REOFFENDED WHICH IS ONE OF THE CORNERSTONES OF THE ALGORITHM WERE TWICE AS LIKELY TO BE CLASSIFIED MEDIUM OR HIGH RISK AND DENIED BAIL, IN OTHER WORDS MORE LIKELY TO SUFFER THE CONSEQUENCES OF BAIL EVEN THOUGH THEY WERE NOT GOING TO REOFFEND. NORTH POINT REPORTED REPORTING THAT DESIGNED THE SYSTEM SO BLACK AND WHITES ARE EQUALLY LIKELY TO REOFFEND BASED ON THEIR SCORES. MIND YOU, THIS ANALYSIS IS PLAYS OUT ACROSS THE RACIAL SPECTRUM BUT MOST CLEARLY VISIBLE IS TENSION BETWEEN BLACK AND WHITES WHITES. AND WHAT WAS AT STAKE IS A REALLY INTERESTING -- ONE WE KNOW STATISTICALLY, IS A QUESTION OF FALSE POSITIVES VERSUS FALSE NEGATIVES. IT'S NOT RESOLVABLE, IT'S A QUESTION OF WHICH IS FAIR. KNOWLEDGE AHEAD OF TIME IN TERMS OF HOW THE MAKE THE DATA PROPORTIONAL VERSUS WHAT HAPPENS ON THE OUTSIDE. THE REASON FOR THIS IS REALLY OBVIOUS. BLACK AND BROWN PEOPLE IN THE UNITED STATES ARE MORE LIKELY TO BE ARRESTED (INAUDIBLE) IT'S ONLY GETTING WORSE. MORE LIKELY TO BE CHARGED HARSHLY, MORE LIKELY TO BE PUB NICHED, MORE LIKELY TO ENTER INTO THE CRIMINAL JUSTICE SYSTEM AND MORE LIKELY TO GET IN TROUBLE IN THE FUTURE BECAUSE OPPORTUNITIES ARE LIMITED. IN OTHER WORDS NORTH POINT IS NOT ASSESSING WHETHER OR NOT PEOPLE ARE ENGAGING IN CRIMINAL ACTIVITY BUT WHETHER OR NOT THEY'RE LIKELY TO BE ARRESTED CHARGED AND CONVICTED. THEY RELY ON BIAS DATA AND PREDICTING OUTCOMES THAT REINFORCE ALREADY BIASED SYSTEM, THEY'RE REAFYING THE STRUCTURE BECAUSE THAT'S THE DATA AVAILABLE TO THEM. WHEN THAT'S THE DATA AVAILABLE TO THEM, THAT'S WHAT THEY ARE USING. AND THE QUESTION IS NOT WHETHER OR NOT THIS IS GOOD DATA BUT A QUESTION IS WHETHER OR NOT WE CAN ACTUALLY DO THE WORK THAT WE WANT TO DO. SO THIS IS A PROBLEM WE'RE SEEING IN PREDICTIVE POLICING. SO SOCIOLOGICAL WORK IS VERY CLEAR ON THAT, WHAT'S MORE LIKELY TO USE AND -- ANY OTHER POPULATION. NOT JUST MARIJUANA BUT EVERYTHING FROM COCAINE, IN THE UNITED STATES BLACKS ARE MORE LIKELY ARRESTED CHARGED AND CONVICTED OF DRUG RELATED CRIMES AND THAT'S WHEN WE FEED THIS BACK INTO THE SYSTEM, WE PREDICT THAT BLACK AND BROWN INDIVIDUALS ARE THE ONE WHOSE ARE GOING TO ENGAGE IN CRIMINAL JUSTICE ACTIVITIES THAT INVOLVE DRUGS. AND PREDICTIVE POLICING TECHNIQUES WE SEND COPS TO DO DRUG ARRESTS TO LOW INCOME COMMUNITIES COLOR PREDOMINANTLY, WE DON'T SEND THEM TO THE FRAT PARTIES THAT FOR WHICH I CAN PROMISE YOU THERE ARE WHITES ENGAGING IN HE WILY SIT DRUG PRACTICES. -- ELICIT DRUG PRACTICES. SO OFTEN NOT KNOWING THEY'RE PRIORITIZING QUALITY OAR EQUITY. WHAT MANY FAIL TO REALIZE IS THEY'RE NOT ACHIEVING E QUALITIED BECAUSE THEY DON'T KNOW THEY LEAT THE DABBING TO DO SO. THEY DON'T KNOW WHO IS NOT IN THE SYSTEM, THEY ONLY LOOK AT WHO IS IN THE SYSTEM. WHEN THE SYSTEMATIC DATA SELECTION PROSIS IS BIASED WE CREATE FEEDBACK LOOPS THAT ARE FUNDAMENTALLY BIASED. WE BIAS THE SYSTEM THE WHOLE WAY THROUGH. BELOSE THE ABILITY -- WE LOSE THE ABILITY TO CORE TO FIGURE OUT WHAT'S GOING ON. THIS IS HOW WE KEEP THE STATUS QUO. HOW DO WE KEEP FROM DOING THAT THE THINGS WE CARE ABOUT THE MOST? MANY MIGHT BE FAMILIAR WITH LA TONYA SWEENEY'S -- THE FORMER CHIEF TECHNOLOGIST OF MACHINE LEARNING HOW THEY WORK IN THE SYSTEMS. SO ONE DAY SHE WAS DOING WHAT MANY OF US DO, EAGLE SEARCH ON GOOGLE. WHAT HAPPENS IF SHE SEARCHES HER OWN NAME AND WHEN SHE SEARCHES SHE NOTICES SOMETHING FUNNY, SHE KEPT GETTING PRODUCTS WHICH WERE CRIMINAL JUSTICE RELATEDS. QUESTIONS ABOUT LA TONYA ARRESTED AND SHE WAS JUST LIKE? SHE HAD NO ARREST RECORD, SHE DIDN'T UNDERSTAND WHY THIS COULD BE COMING UP SO SHE DECIDED AS COMPUTER SCIENTIST SHE WOULD TRY TO FIGURE THIS OUT. SO SHE GOOGLED BABY NAMES IN THE UNITED STATES AND CORRELATION BETWEEN BABY NAMES AND RACE OVER 20 YEARS AND THROUGH THEM AT GOOGLE JUST TO SEE WHAT KIND OF ADS SHE RECEIVED IN RETURN. NOT SURPRISINGLY THE NAMES WERE FAR MORE LIKELY TO RECEIVE CRIMINAL JUSTICE RELATED THAN WHITE BABY NAMES. SHE KNOWS THAT GOOGLE IS NOT SELLING INFORMATION BASED ON THE RACE OF NAME. SHE KNOWS THAT. BUT WHAT SHE DOES KNOW IS THE WAY THEY'RE ADS INFRASTRUCTURE WORKS IS IT REALLY DEPENDS ON WHO CLICKS ON WHAT AD. AND IT TRAINS THE DATA. THAT BASICALLY EVOLVES BECAUSE OF COURSE GOOGLE WANTS TO MAKE SURE ADS ARE PUT IN A POSITION PEOPLE ARE MORE LIKELY TO CLICK ON THEM, THAT'S HOW MONEY IS MADE. SO WHAT HAPPENS IS THAT THE POPULATION AS A WHOLE, PEOPLE WHO ARE USING GOOGLE, WHEN THEY SEARCH FOR PEOPLE WITH PREDOMINANTLY BLACK NAMES WERE FAR MORE LIKELY TO CLICK ON CRIMINAL JUSTICE RELATED PRODUCTS THAN WHEN THEY SEARCH FOR WHITE NAMES. IN OTHER WORDS, THEY TRAIN GOOGLE HOW TO BE RACIST IN AMERICA. AND GOOGLE IS ABLE TO FEED THAT RIGHT BACK AT THEM. ALL THIS IS DONE NOT BECAUSE GOOGLE DESIGNED IT TO BE THAT WAY BUT BECAUSE GOOGLE DESIGNED IT TO LEARN FROM THE FEEDBACK LOOPS AND WHEN WE DESIGN TO LEARN FROM FEEDBACK LOOPS WE LEARN SOCIETAL PREJUDICE AND THAT IS THAT PROCESS THAT WE SEE AND WE DON'T HAVE A WAY NECESSARILY ACCOUNTING FOR IT. PART OF IT IS CATEGORIZATION IN AMERICA IS EXTREMELY FRAUGHT. NO ONE KNOWS THIS BETTER THAN CENSUS. IT'S ALWAYS DELIGHTFUL TO SEE CENSUS WORK SO HARD TO FIGURE OUT THESE DYNAMICS. THERE'S A LONG HISTORY OF TRYING TO FIGURE HOW TO CATEGORIZE PEOPLE AND ACTUALLY LOVE IT, ONE OF MY FAVORITE THINGS TO SEE HISTORY OF CENSUS AND TRYING TO FIGURE RACE IN AMERICA. IT'S NOT SOMETHING EWE UNIQUE FOR THE UNITED STATES. A BOOM FOR DATA SCIENCE IS A BOOK CALLED "SORTING THINGS OUT" THE POLITICS OF TRYING TO CATEGORIZE AND ABOUT APARTHEID SOUTH AFRICA AND RIPPLE EFFECTS WHEN WE TRY TO CATEGORIZE PEOPLE AND IT WENT TERRIBLY AWRY, FAMILIES SPLIT APART WHEN THEY HAD CHILDREN DARKER THAN THEMSELVES, IT'S CHANGED EVERYTHING FROM HALLOWEEN POLICY, ET CETERA, WE CAN LOOK AT APARTHEID AND SAY IT WAS A HORRIBLE SITUATION, REALLY GLAD WE'RE DONE WITH IT. AT THE SAME TIME KEEP IN MIND THAT'S THE REALITY OF AMERICAN SYSTEMS AS WELL. SO FAMILIES SPLIT APART IN THE UNITED STATES FOR A LONG TIME BECAUSE OF HOW WE DEAL WITH RACIAL CATEGORIZATION, AND ANTI-SEMIATION LAWS IN THE U.S. ARE THE SAME SET OF LOGIC. THE CATEGORIES THAT SET USE THIS DAY SHAPE A LOT OF POLITICS AND POLICIES HOW WE THINK ABOUT RACE. IT ALSO AFFECTS ECONOMICS AND WAY POLITICS THINK JERRYMANDERING NOT TO MENTION THE ILLEGAL PRACTICES OF RED LINING STILL GO ONFUL >> WE HAVE ABOUT 15 MINUTES. WANT TO MAKE SURE WE HAVE TIME >> SURE. GOUT. KEEP IN MIND THAT IN THE UNITED STATES HOW NATIVE AMERICAN TRIBES RECEIVED THE SCHOOL SUPPORT DEPENDING HOW PEOPLE CATEGORIZED THEM AS INDIANS IN THE CENSUS. THE PROBLEM WITH DATA ANALYTICS, OR I SHOULD SAY ONE OF THE BEST SENSIS THEY HAVE DONE A GOOD JOB TRYING TO BATTLE THESE THINGS THROUGH BUT AT THE SAME TIME IN INDUSTRY WE RUN INTO IT ALL THE TIME, WE DON'T THINK ABOUT THE RIPPLE EFFECTS, WE THINK ABOUT COMBINING DATA FROM INDUSTRY WITH COMBINED DATA WITH POLITICAL DATA WE HAVE TO THINK HOW THIS HAS BEEN A FRAUGHT PROCESS. CONTEMPORARY DATA ANALYTICS BASICALLY ABLE THINK ABOUT MACHINE READABLE DESCRIPTORS. ONE THING THAT'S FASCINATING IS THE FEDERAL TRADE COMMISSION WAS INVESTIGATING DATA BROKERS AND FOUND SOME DATA BROKERS THRIFTY ELDER AN URBAN SCRAMBLE BUT MOST DATA ANALYSIS IS NOT WORKING THIS WAY,'S NOT EXPLICITLY PREJUDICIAL, MOST DATA ANALYSIS MAKINGS IT AS PART OF CLUSTERING WITHOUT UNDERSTANDING THE PEOPLE OR PROPERTIES THEY USE, JUST MATH, BUT THE MATH AND DECISIONS MADE BY THE MATH HAVE SERIOUS RAMIFICATIONS. SO LET'S TALK ABOUT WHAT'S HAPPENING NOW IN THE JOBS ECOSYSTEM. IF YOU WANT A JOB AT A COMPANY LIKE WAL-MART YOUR RESUME IS FILTERED THROUGH THIRD PARTY APPLICATION TRACKING SYSTEMS. AND ANALYZED SOON HOW RESUME ARE MATCHING AGAINST OTHER AT THAT JOB AS WE TALK BEFORE MATCHING TO ALREADY PROBLEMATIC DATA BUT ALSO MEANS THAT WHAT IS HAPPENING IN THESE KINDS OF THINGS IS WE DECIDE TO REMOVE CATEGORIES THAT ARE PROTECTIVE CATEGORIES, RACE, GENDER, ET CETERA. THE SYSTEMS RELEARN IT AND ONE OF THE THING WE'RE SEEING IN THESE SYSTEMS IS THEY BASICALLY -- HERE IS WHERE I THINK THERE'S SOMETHING BEAUTIFUL GOING ON BECAUSE THERE'S A WAY OF ACTUALLY REMEDYING THIS, TECHNICAL REMEDY FOR THIS, SO (INAUDIBLE) COLLEAGUES MATHEMATICALLY RENORMALIZE DATA TRAINING SETS AS WAY OF GETTING AROUND THIS WHICH IS A PRETTY PHENOMENAL INTERVENTION, IT'S NOT NECESSARILY A WAY OF PERFECTLY DEALING WITH BIAS BUT THE WAY OF STARTING TO GO AFTER IT SO I REALLY RECOMMEND PAYING ATTENTION TO A WHOLE COMMUNITY OF PEOPLE OF FAIRNESS ACCOUNTABILITY AND TRANSPARENCY AND MACHINE LEARNING AND COMPUTER SCIENCE LOOKING FOR TECHNICAL INTERVENTIONS TO THIS. BUT AS THEY LOOK FOR THESE INTERVENTIONS WE'RE ALSO SEEING HOW THESE PROBLEMS HAVE BEEN COME UP OVER AND OVER AGAIN. I WANT TO CONCLUDE BY GIVING A FEW MOMENTS ON ACCOUNTABILITY. BECAUSE AS WE MOVE TOWARDS OPEN DATA AND MORE SOPHISTICATED ALGORITHMS WE NEED TO EXPLICITLY STATE OUR VALUES AND CRAFTING AND HOLDING THOSE THINGS ACCOUNTABLE. ACCOUNTABILITY IS NOT SIMPLE, ONE BIG PROBLEM NOW IS WE DON'T KNOW, HAVE TOOLS TO DO ACCOUNTABILITY WELL, THIS IS WHERE I LAUGH WHENEVER FOLKS ASK US AS AN INDUSTRY TO DO THINGS WELL BECAUSE THERE'S SO MANY GAPS OF WHAT WE'RE DOING IN INDUSTRY AROUND THESE ISSUES. WE'RE NOT THERE YET. WE DON'T KNOW HOW TO CHECK AROUND WHAT THESE SYSTEMS. GOOGLE DIDN'T DESIGN FOR PREJUDICE, THEY DON'T KNOW HOW TO FIX IT, FACEBOOK ISN'T COMBINED FOR CONSPIRACY THEORISTS BUT THEY'RE GETTING USED THAT WAY. WAL-MART DIDN'T DESIGN PARTY VENDORS TO ENGAGE IN THESE PRACTICES BUT THEY'RE DOING IT. GOVERNMENT IS DIFFERENT THAN CORPORATIONS IF YOU GET A RIDICULOUS ADVERTISEMENT, YOU'LL LAUGH, I LOVE BEING LABELED A TRUCKER BUT EAT NOT FUNNY WHEN PEOPLE ARE INCOURSE RATED. IT'S NOT FUNNY WHEN THE ABILITY TO HAVE FREEDOM AND FLEXIBILITY TO GET A JOB IS CURTAILED. THAT'S WHAT THE GOVERNMENT IS HERE TO PROTECT. MAKE CERTAIN WE ACTUALLY ADDRESS THESE BIASES IN A CRITICAL WAY. HERE IS WHERE I WANT TO TELL YOU ABOUT THE WORK OF MADELINE ELLIS FOR A SECOND. SHE WAS STUDYING AUTO PILOT AVIATION AND THE FAA HAD A SET OF CONVERSATIONS IN THE 1970s AROUND THIS, BUT HOW IMPORTANT IT WAS TO KEEP A HUMAN IN LOOP IN CASE OF AN EMERGENCY. MOST OF YOU FLY ON A REGULAR BASIS, YOU PROBABLY KNOW EVEN IF YOU DON'T WANT TO THINK ABOUT IT THAT IT IS FLOWN BY AUTONOMOUS SYSTEM BUT THE PILOT JOB IS TO STEP IN IN CASE OF EMERGENCY. WHAT IT MEANS IS WE HAVE SOMEBODY WHO IS SITTING IN A COCKPIT WHO IS SYSTEMATICALLY DESKILLED ON THE JOB, HASN'T REALLY FLOWN A PLANE IN QUITE A LONG TIME WHO IS SUPPOSED TO STEP IN THE HARDEST PART OF THE PROBLEM. THAT'S A SCARY SITUATION. AND WHAT THAT MEANS IS THAT THAT PERSON'S REAL PURPOSE IS TO BE A LIABILITY SPONG. THEY ARE THERE TO SUCK UP THE LIABILITY WHEN EVERYTHING GOES AWRY. AND MADELINE REFERS TO THIS AS A MORAL CRUMPLE ZONE, THE MOMENT WE'RE KEEPING HUMANS IN THE LOOP SO THEY CAN PLAY THE ROLES OF CRUMPLED ZONE. THINK ABOUT YOUR CAR. WHAT DOES IT MEAN WHEN WE THINK THIS WAY, IN THE REALLY THINKING ETHICS OF LONG TERM COMPLEX SYSTEMS? THAT'S WHAT I WOULD ARGUE WE NEED TO CONCLUDE AS WE THINK ABOUT ACCOUNTABILITY. WE NEED ACCOUNTABILITY IN TERMS OF INTERACTIONS OF LARGE SCALE SYSTEMS. IT'S NOT ABOUT INDIVIDUAL ACTORS, IT'S UNDERSTANDING THE INNER PLAY BETWEEN THEM, THE RESISTANCE, MANIPULATION, GAMES THAT ARE PLAYED, THE POLITICAL AGENDAS, THE ISSUES OF INEQUALITY THAT COME INTO BEING. AS WE THINK ABOUT THE ACCOUNTABILITY SERIOUSLY WHICH IS SOMETHING THE GOVERNMENT CAN DO, WE CAN COME UP WITH A BETTER WAY TO ACTUALLY MAKE CERTAIN WE DEAL WITH OUR DIGITAL FUTURE. THANK YOU. >> THERE WERE NO ISSUES AT ALL BAKED INTO -- YOU DEFINITELY HAVE A INTENSERRINGS CONVERSATION, I LOVE THE FACT THERE'S SO MANY PEOPLE AT THIS TABLE THAT CAN REACT TOO THE ISSUES THAT YOU BROUGHT UP. SO WHO WILL JUMP IN FIRST? >> COME ON, GUYS, YOU ARE NOT THAT QUIET. >> FIRST THING, I THINK IT'S A GREAT INTERESTING CONVERSATION. I'M WONDERING -- MAYBE TWO THINGS ONE, BASED ON CONVERSATIONS TODAY, BRING SOME OF THAT DISCUSSION, HOW DO YOU -- HOW WOULD YOU FIRST START THINKING ABOUT ADVISING COMMERCE ON HOW THEY CAN ACTUALLY TAKE THESE IDEAS AND TURN THEM INTO SOME ACTION WITHIN THE COMMERCE DATA SERVICE? AND TWO, AS YOU WERE TALKING I'M COMING BACK TO THE IDEA, YOU HAD SEGMENTATION IN GOVERNMENT. EVERYONE HAS THEIR ROLE. SOMETIME THAT WAS GOOD, SOMETIMES BAD BUT SHOULD WE THINK ABOUT COMMERCE DATA OR ANY TYPE DATA SERVICE AS BEING SOMEWHAT DISTINCT? SO THEY HAVE A GOAL AND THEIR MISSION IS TO MAKE DATA PUBLIC OR MAKE -- THE GUY WORK WELL, AND AS MUCH AS WE CARE ABOUT THESE OTHER ISSUES, DO WE SAY, WELL, THE ULTIMATE PURPOSE OF WHATEVER DATA SERVICE IS STILL NOT TO NECESSARILY ACHIEVE THESE OUTCOMES. WE HAVE POLICY FOLKS WHO ARE ON THAT, EDUCATION FOR EXAMPLE, NOT HAVING SCHOOL CHOICE, WE CHANGE VACATION POLICY, WE DON'T CHANGE TRANSPARENCY IN SCHOOLS SO HOW DO WE RECONCILE THOSE TWO IDEAS? >> FIRST QUESTION, I THINK ONE OF THE THINGS THAT NOT ALWAYS VISIBLE ABOUT INDUSTRY, THOSE OUTSIDE OF IT, IS HOW MANY DIFFERENT GROUPS OF PEOPLE EXIST TO TRY TO BREAK THE SYSTEM? WHETHER WE'RE TALKING HACKERS IN A SECURITY SENSE, PEOPLE WHO ARE THINKING ABOUT ADVERSARIES, THAT LOGIC HAS TO BE BAKED INTO THE PROCESS. ONE OF THE CHALLENGES IS WE MAKE VISIBLE THE PROCESS OF AWESOME IMAGINATION, BUT STRUCTURES THAT HAVE TO BE PUT INTO PLACE TO THINK ABOUT WHERE EVERYTHING CAN GO WRONG AND OFTEN NOT THE SAME PEOPLE BECAUSE IT'S HARD TO HOLD THOSE PEOPLE IN THE SAME POSITION. THE FOLKS WITH THE IMAGINATION THE BIG EYES, THEY HAVE TO BE LET'S DO IT, THIS COOL THING HERE, THAT'S GREAT. AND PLEASE GO AHEAD, THAT'S FABULOUS BUT YOU NEED OTHER STRUCTURES THAT ARE YOUR BACKBONE AND NOWHERE IS THAT MORE IMPORTANT THAN GOVERNMENT AGENCY. YOU NEED A HACKER LINE EXISTING ALONGSIDE THE PERSON BUILDING SECURITY. AND THE SAME IS TRUE FOR THESE KINDS OF SOCIAL ISSUES. IN TERMS OF TRANSPARENCY, I THINK PART OF IT IS YOU HAVE TO CONTEND WITH WHAT IS THE OUTCOME OF TRANSPARENCY THAT WE'RE HOPING TO ACHIEVE? AND WE WERE TRANSPARENCY SITTING ALONE DATA ACCESS OPEN DATA ALONE MAY WORK AGAINST THE GOALS WE'RE TRYING TO GET THERE. WHEN YOU'RE ROLLING OUT THE PROCESS OF MAKING DATA AVAILABLE, HOW ARE YOU ALSO MAKING SURE EVERYTHING FROM POLICIES TO THE SKILLS TO BE ABLE TO USE THAT DATA, TO THE DYNAMICS BUREAUCRATICALLY AROUND ARE ALIGNED IN A WAY IT'S NOT GOING TO BE MISUSED. REALIZING ONE OF THE THINGS THAT'S FUNNY ABOUT INDUSTRY IS BULLET START UPS PLAY WITH DATA, AND NO ONE CARES. BECAUSE THEY'RE OFTEN EXPERIMENTING AND GROW UP AND BECOME BIG COMPANIES AND THEY THINK OF THEMSELVES AS START UP AND THAT'S WHERE FRAUGHT ISSUES HAPPEN. COMPANIES BIG COMPANIES AROUND THIS TABLE, WE CAN'T ROLE SOMETHING OUT HOPE IT WILL BE USED WELL BECAUSE THINGS GO TERRIBLY AWRY, WHEN YOU START WITH -- THE GOVERNMENT STARTS WITH SCALE. YOU DON'T GET THE PRIVILEGE OF LITTLE EXPERIMENT HERE. YOU HAVE THE MEDIA IN YOUR FACE. YOU HAVE AD SORE SAYS IN YOUR FACE. YOU HAVE TO THINK THROUGH RATHER THAN MOVING AT THE SPEED START UP CAN THINK LIKE. THIS IS -- NO, DON'T, BECAUSE THEY BREAK SYSTEMS AND IT'S OKAY WHEN IT ALL FAILS. IT IS NOT OKAY WHEN COMMERCE FAILS, SO I THINK A LOT IS UNDERSTANDING WHAT IS THE RISK YOU'RE WILLING TO TAKE AND WHAT IS THE FAILURE THAT U YOU'RE WILLING TO ACCEPT. IF THAT FAILURE MEANS PEOPLE GOING TO JAIL THAT'S NOT OKAY. IF THAT FAILURE MEANS PEOPLE DYING DUE TO WEATHER EXPERIMENTS, THAT IS NOT OKAY. THAT'S UNDERSTANDING WHAT THE DYNAMICS THERE ARE ARE SO CRITICAL TO THIS PROCESS SO THE IDEA WITH MOVING FAST MY NOT BE GOOD ENOUGH. >> I THINK WE'LL STAN, BILL, ALAN. >> THAT WAS THOUGHT PROVOKING. ON SEVERAL FRONTS. KUDOS FORGETTENING THAT AMOUNT OF TIME. THINKING OF SO MANY POINTS. I GUESS I INTERPRET WHAT YOU DESCRIBED, THE -- WHERE I WAS HEARING MAYBE IN YOUR MARKETS OPEN DATA IS DANGEROUS, I CAN SEE IT IS, BUT ALSO IT SEEMS LIKE WHAT I TOOK AWAY IS THINKING IT'S IMPORTANT TO THINK ABOUT OPEN DATA AS JOURNEY, NOT AS A DESTINATION. BUT IT'S STILL A GOOD JOURNEY AND THE FACT YOU CAN'T GET IT RIGHT ON THE FIRST ITERATION, WE SHOULD BE -- THAT'S STILL OKAY THAT AT LEAST THE WHAT MOTIVATES OATH DATA INITIATIVE IS THE FACT THERE IS A TRUTH OUT THERE AND IT WILL GET TO THAT TRUTH FASTER THROUGH TRANSPARENCY AND YOU'RE RIGHT THEY'RE LIKE IN YOUR SCHOOL EXAMPLE, YOU MAY MAKE INITIAL THINGS WE'RE LOOKING AT SCHOOL RANKINGS, EXPLICITLY AND WE DON'T FACTOR IN THE FACT THOSE RANKINGS FACTOR WHO LIVES IN THAT AREA, THE NEXT GENERATION SOMEBODY LOOKS AT THE VALUE ADD OF SCHOOL, MAYBE NOT PERFECT EITHER BUT FEELS LIKE IT SETS IN PLACE ITERATION CYCLE WHERE WE END UP IN A BETTER PLACE. AND I WOULD THINK LIKE IT CREATES A MARKETPLACE OF IDEAS WHERE THEN PUT INCOME BACK AND SAY HEY YOU CREATED THIS PREDICTIVE ALGORITHM WHICH IS RELEARNING A BIAS BECAUSE -- THAT ENDS UP CREATING A GOOD ITERATION WHERE YOU FIGURE OUT HOW YOU REWEIGHT THAT DATA SET WAL-MART FOR EXAMPLE, LESS RACIAL ETHNIC BIAS PREDICTIVE MODEL. >> FIRST LET'S BE CLEAR, WE'LL REVERSE ENGINEER IT, IT ENCOMPASSES BY NO MEANS PUBLIC AND ONE PROBLEM IN CRIMINAL JUSTICE IS NONE AL THERE'S NO ALGORITHMIC TRANSPARENCY AT ALL. BUT IN EARLIES THE OF ASPECTS THE QUESTION IS WHAT IS THE ITERATION? WE KNOW TECHNICAL ITERATION CYCLE IS FASTER THAN POLITICAL ITERATION CYCLE. THAT'S PART OF THE PROCESS SO WHAT DO WE NEED THE MAKE CERTAIN WE DON'T SCREW UP BECAUSE OF POLITICAL -- SO MUCH SLOWER. SO GOOD EXAMPLE, JUST MENTION HOUSING, RIGHT? WHAT'S HAPPENING IN HOUSING IS THAT OPEN DATA MOVEMENTS HAVE BEEN USED FAR MORE BY DEVELOPERS AND HAPPEN BY ADVOCATES TRYING TO MAKE CERTAIN HOUSING IS FAIR, THE RESULT WE'RE SEEING HUGE RISE IN EQUALITY IN TERMS OF HOUSING IN URBAN ENVIRONMENTS. I'M WATCHING IN THE WAY A LOT OF OPEN DATA THAT YOU GUYS HAVE BEEN PROVIDING, HAVE ACTUALLY MADE IT SO MAJOR LANDLORDS ARE GOING AFTER AND USING THAT DATA TO REALLY PUSH PEOPLE PAST THE POINT OF ECONOMIC BRINK. SO HOW DO WE MAKE CERTAIN -- AND THE ANSWER OF COURSE IS WE SHOULD BE PROTECTION. AND THAT OF COURSE IS POLITICAL. IF GOAL IS PROTECTION IN HOUSING TO MAKE SURE WE'RE OKAY BUT THE GAP IN BETWEEN CAN BE SO COSTLY WHO ARE WE WILLING TO LET SUFFER IN THAT PROCESS? MAYBE HOUSING IS EPIISABLE IN JUSTICE BUT PART IS THAT'S A SET OF MORAL DECISIONS AND ETHICAL DECISIONS THAT WE NEED TO WORK OUT BECAUSE OTHERWISE WE'RE PAYING THE COST OF INEQUALITY DOWNSTREAM BECAUSE WE'RE HOPING THAT WE'RE GOING TO TRIGGER AN EXPERIMENTAL PROCESS THAT'S NOT CLEAR WE'RE GOING TO MOVE IT AT THE SPEED ACCEPTABLETOR ACHIEVING OUR GOALS. >> IN THE EXAMPLE YOU GAVE IT FEELS LIKE OPEN DATA IS MOVING -- YOU'RE RIGHT THAT THE RECOURSE, A LOT OF DATA IS USED FOR ARGUMENTS ABOUT POLITICAL HOUSING AND RENTAL HOUSING WHERE WE DO A LOT OF WORK AND PROVIDE MUNICIPALITIES BUT DATA ABOUT LACK OF HOUSING WHAT YOU SHOULD DO AIN'T, THE RECOURSE IS POLITICAL, WHEREAS FIRST CYCLE IS THE COMMERCIAL ONE AND THOSE MOVE DIFFERENT PACES BUT HAVING OUT THERE, SAY WELL, OKAY IN ORDER TO COMBAT THIS POLITICAL HOUSING CRIES IS I WILL GIVE MORE FUNDING LOW INCOME HOUSING TAX CREDIT FUNDING OR HOUSING VOUCHERS, IT ENDS -- I GUESS MY FEELING IS HAVING IT OUT THERE, ULTIMATELY WILL END UP IN A BETTER PLACE. WHAT YOU'RE SAYING IS THAT GETTING THERE CAN BE MAYBE MESSY WHICH IS TRUE, BUT IT DOES FEEL LIKE WE ULTIMATELY END IN A BETTER PLACE THAN IF WE DIDN'T HAVE THE DATA AT YOU WILL ALL. >> WHERE THE HYPOTHESIS IS, WE HOPE THAT WE WILL GET TO A BETTER PLACE. THE EDUCATION DATA TO ME IS DISTURBING IS THAT WE'RE 10, 15 YEARS INTO IT AND THINGS ARE CONTINUING TO GET WORSE RATHER THAN BETTER. I DON'T KNOW, I WANT TO BELIEVE YOU, I WANT TO BELIEVE WE'LL GET TO A BETTER PLACE TO EMPOWER PEOPLE TO FIND WAYS OF EVENNESS BUT GROW A LOT OF INEQUALITY TO GET THERE, QUESTION IS CAN WE STOMACH IT AND WHO PAYS THE COST? THIS IS ONE OF THE REASONS I SEE MORAL TRADE OFF. AND I THINK THAT IT'S A MOMENT THERE'S CERTAIN AREAS I THINK WE CAN DO MORE EXPERIMENTATION WITHOUT SERIOUS SOCIAL COST. I THINK WE DON'T HAVE THE PROCESSES TO HAVE THAT CONVERSATION HOLISTICALLY. >> YOU FEEL THAT IT'S DATA THAT'S GOTTEN TO A WORSE PLACE OR THE FACT DATA REVEALED PROBLEMS IN THE POLITICAL PROCESS AND NOT CAUGHT UP IN PROBLEM? >> DATA -- PEOPLE ALWAYS USE GOSSIP TO FIGURE HOW TO NEGOTIATE THE SAME SORTS OF THINGS. THE DATA AND THE USE OF THE DATA BY COMMUNITIES, HAVE ACTUALLY MADE THINGS WORSE. AND THAT IS WHAT I STRUGGLE WITH. >> I THOUGHT IT WAS A EXTRAORDINARY PRESENTATION. I'M ONE OF THESE I'M FOCUSED ON THE -- DANAH IS RIGHT THIS IS A MIND OPENER FOR ME. WHAT WORRIES ME IS THE FACT DATA DECISION PROCESSES ARE GOING TO BE FASTER. BUILDING SYSTEMS THAT MAKE IT FASTER SO THESE PREJUDICES HAVE BEEN PREJUDICE DERIVATIVES WE MAY NOT SEE UNTIL THE LAST MINUTE. I WON'T EVEN KNOW LEVEL OF PREJUDICE BECAUSE THE SYSTEM MOVES FASTER RATE SO WE HAVE TO THINK ABOUT THIS NOW. Z I THINK IT'S A VERY INTERESTING POINT. I WAS LOOK AT SECURITIES ONE DIMENSION, YOU CAN PLAY WITH MANIPULATION BUT MACHINES PREJUDICED MULTIPLE DEGREES ARE SOMETHING TAKE ON AT DIFFERENT LEVEL. THIS IS A GREAT WAY TO HAVE THAT HERE. >> I THINK IT'S A TREMENDOUS PRESENTATION, WHEN I TALKED TO DANAH BEFORE ABOUT THIS UNRELATED ISSUES, IT'S THOUGHT PROVOKING. SO FOR ME, MY BACKGROUND WAS I DID PRO BONO CRIMINAL DEFENSE, A LOT OF THINGS CRIMINAL JUSTICE HITS AN ISSUE I CARE ABOUT, I CAN ADD OTHERS TO YOUR LIST FOR EXAMPLE, WHEN SOMEBODY IS IN JAIL ON BAIL, -- NOT ON BAIL, MOST OF THE TIME WHEN THEY HAVE A CONVERSATION, IN FACT VIRTUALLY ALL THE TIME, EVERY CONVERSATION IS RECORDED, AND YOU WOULD BE SHOCKED ABOUT EVEN IF YOU KNOW THAT IT'S RECORDED AND CAN BE USED IN COURT AGAINST YOU, HOW MANY PEOPLE SAY THINGS THAT ARE INKNOCK WOWS BUT SOUND REALLY NOT INNOCUOUS WHEN THAT HAPPENS? SO THAT'S A FUNDAMENTAL ASYMMETRICAL CONSEQUENCE OF NOT GETTING BAIL THAT IS REALLY NOT VERY WELL KNOWN. AND IT COMES BACK TO CONVICTIONS THAT NOT SURE WOULD HAVE HAPPENED HAD THEY NOT BEEN LOCKED UP BUT IN THE CRIMINAL JUSTICE SYSTEM THE ADMISSIBILITY OF ANY DATA OR STATISTICS OR WHEN A JUDGE IS GOING TO CONSIDER THAT, IS REALLY A WHOLE OTHER BALL OF WAX. SO IT RAISES SOME REALLY INTERESTING ISSUES FOR ME. WHEN I THINK I WANT TO MAKE SURE OUR EXPERTS ARE TALKING BUT WHEN WE LOOK AT OPEN DATA, A LOT OF THE REASON I LIKE THIS CONVERSATION AND I THINK IT'S IMPORTANT IS A LOT OF WHAT WE HAVE SEEN ON THE POLICY SIDE IS THE OPEN DATA MOVEMENT IS ABOUT TRANSPARENCY, SOME OPENNESS, THE FOIA MOVEMENT, TRYING TO AUTOMATE THE MOVEMENT SO ONE COMES IN YOU HAVE O TO GET DATA OUT WE GET REQUESTS AND WANT TO MAKE THEM AVAILABLE. ONCE WE DEIDENTIFY DATA THERE'S A GOAL TO DRIVE OPENNESS AND USE OF IT AND DANAH IS ASKING INTERESTING QUESTIONS BUT GLAD WE'RE ACTUALLY CONSIDERING SO WHY DON'T WE GO TO PHIL. >> I WANT TO ASK YOU ABOUT A CLOSE RELATED BUT DIFFERENT PROBLEM THAT WE MAY CALL -- SHOULD WE TRUST PEOPLE WITH DATA AND I HAVE A REAL WORLD EXAMPLE . FEW YEARS AGO NO ADMINISTRATIVE WAS TESTIFYING IN CONGRESS IN TOPIC OF TORNADO WARNING CAME UP AND TYPICALLY TORNADO WARNING LEAD TIMES ARE TEN AND 20 MINUTES SO SHE WAS ASKED WELL, IF WE CAN EXTEND TO 60 MINUTES WOULD THAT BE A GOOD THING? SHE GAVE A VERY INTELLIGENT BUT NUANCED ANSWER WHICH IS THAT AFTER SOME TIME PERIOD, 20 MINUTE, 30 MINUTES PEOPLE NO LONGER THINK IMMINENT SO THEY GO BACK TO PLAYING VIDEO GAME AND SAY I CAN RESPOND LATER SO THEREFORE, YOU HAVE TO BE ABLE TO GIVE THE INFORMATION TO PEOPLE IN A WAY THAT THEY TAKE THE RIGHT ACTION ON. AND SHE GOT HER HEAD TORN OFF BY ONE OF THE CONGRESSMEN WHO SAID YOU MEAN YOU DON'T TRUST PEOPLE WITH INFORMATION? THERE'S A REAL DILEMMA HERE BECAUSE AT SOME LEVEL HE'S RIGHT, AT SOME LEVEL HE'S WRONG. YOU HAVE TO DELIVER INFORMATION TO PEOPLE IN A WAY THEY MAKE USE OF IT, THAT'S SAME AS NOT TRUSTING PEOPLE WITH INFORMATION. >> I THINK YOU'RE TOTALLY RIGHT. I THINK A BEAUTIFUL EXAMPLE, WEATHER. HURRICANES YOU KNOW BETTER THAN ANYBODY THAT WE CAN HAVE PREDICTIONS OF HURRICANES MANY, MANY DAYS OUT. BUT IT'S HUGE VARIANTS. SO THE QUESTION IS, IS THE CLOSER YOU GET TO THE EVENT THE MORE YOU'RE PROBABLY MODELS ARE CLOSER TO REALITY. BUT HOW DO YOU INFORM IT? ARE YOU GOING TO -- IT'S GOING TO HIT AND YOU'RE LIKE -- WE HAVE FULL 24/7 HURRICANE. WHERE EVERY DEVIATION IS (INDISCERNIBLE) THE EXPERTS ARE WORKING WITH PROBABLYISTIC SYSTEMS. AND THAT'S EXACTLY THAT MOMENT PROBABILISTIC. WHERE WE CAN MAKE THIS DATA, BUT QUICKLY GETS POLITICIZED. IT QUICKLY GETS TURNED INTO A MONEY MAKING NARRATIVE, POLITICAL EVENT NARRATIVE WHEN AT THE END OF THE DAY, TRYING TO MAKE DATA AVAILABLE TO SAVE LIVES TO SAVE PROPERTY AND THINGS THAT DON'T WORK WELL. WHAT IS THE PURPOSE -- HOW TO NAVIGATE IT? THIS IS WHERE WE STRUGGLE WITH WE LIVE IN A SOCIETY WHERE WE CAN ACCESS TONS OF INFORMATION BUT WE (INDISCERNIBLE) TO MAKE SENSE OF ALL THE INFORMATION. I DON'T HAVE THE SENSE TO FIGURE OUT HEALTHCARE PA CAM. -- PACKAGE. THE MORE INFORMATION HOW WE DEAL WITH INFORMATION SKILL SETS THAT PEOPLE HAVE (INAUDIBLE) SO YOU'RE RIGHT. (INDISCERNIBLE) AND IT ENDS UP (INDISCERNIBLE) (INDISCERNIBLE) (INDISCERNIBLE) (INDISCERNIBLE) FOR THE POPULATION AS A WHOLE, INFORMATION -- (INDISCERNIBLE) (INDISCERNIBLE) (INDISCERNIBLE) CAN BE HUGELY CHALLENGING. AND -- WHAT HAPPENS WITH THE DATA SET BEING OUT THERE, WHERE IT'S READ FOR FUNDAMENTALLY DIFFERENT POLITICAL REASONS IN DIFFERENCE WAYS BECAUSE THE QUESTION OF WHETHER OR NOT YOU'RE GOING TO BE WILLING TO ACKNOWLEDGE THE DATA THAT'S MISSING AND HAVE A LOT TO DO WITH WHAT YOU'RE TRYING TO MOVE FORWARD. (OFF MIC) >> COMMERCE COULD ADOPT HAVING BASICALLY BEFORE RELEASING A PIECE OF DATA OPENLY. AT LEAST DONE SUBSTANTIAL WORK IN TRYING TO EVALUATE OTHER USES TO WHICH IT CAN BE PUT OR MAKING SURE THAT YOU'RE WORKING THROUGH AS CENSUS DOES, WORKING THROUGH TO REMOVE BIASS FROM THE DATA SO WHEN IT'S PICKED UP IT'S HARD TO USE IN UNPREDICTABLE WAYS. >> THERE IS MUCH TO BE LEARNED FROM THE COMMERCE DEPARTMENT BACK TO US AS INDUSTRY. I ACTUALLY THINK THERE'S CERTAIN AREAS THEY STRUGGLE WITH THIS FOR DECADES TRYING TO FIGURE HOW TO CLEAN DATA RESPONSIBLY, TRYING TO FIGURE HOW TO MAKE THAT DATA FOOLPROOF REALIZING THE CENSUS IS IMPERFECT BUT TRYING TO UNDERSTAND SCIENTIFICALLY WHAT THAT MEANS. ALL THAT WORK AND I HAVE A FEELING THAT FRANKLY WE'RE NOT AS GOOD IN INDUSTRY AS THEY ARE. >> THE NEXT -- -- IF I MAY, REALLY THANK YOU TO DANAH. I THINK THIS IS GOING -- GOT TO BE PART OF OUR LONG TERM CONVERSATION AND WE HAVE TO THINK ABOUT THE ISSUES THAT YOU RAISED AND HOW WE DO DATA SETS SO I REALLY APPRECIATE IT AND ALSO FOR SOMEHOW SPEAKING FOR 30 MINUTES WITHOUT BREATHING AT THE BEGINNING OF A SESSION, WHICH IS REALLY A CHALLENGE. ALL RIGHT. DANAH, THANKS, AND THANKS TO EVERYBODY WHO PARTICIPATED IN THAT PRESENTATION. OKAY. SO OUR NEXT SESSION, ARE WE ABLE TO GET JEFF MEISEL IS HE PARTICIPATE SOMETHING I CAN'T REMEMBER. >> SHOULD BE ABLE TO GET HIM ON THE SCREEN. HE SHOULD BE UP THERE ANY MINUTE. >> HOW ABOUT THAT. LET ME SAY JUST A WORD OR TWO BEFORE I TURN IT OVER TO DANIEL. TO RECOGNIZE, JEFF, I'M NOW DOING THIS REGULARLY BUT I WANT TO MAKE SURE THE CREDIT GOES WHERE IT'S DUE. WE HAVE LEADERS IN JEFF CHEN AND JEFF MEISEL WHO HAVE REALLY STEPPED UP AND CURATED AND DRIVEN OUR WORK IN REALLY TERRIFIC WAYS. AND ONE OF THE WAYS THAT WE HAVE INSTITUTIONALIZED, REALLY WORK TO INSTITUTIONALIZE THIS AND WHERE I FEEL VERY COMFORTABLE HAVING JEFF REALLY -- JEFF CHEN WILL BE TAKING OVER AS DEPUTY CHIEF -- IS DEPUTY CHIEF DATA OFFICER BUT WILL SOON BE ACTING CHIEF DATA OFFICER THROUGH THE TRANSITION. AND BRAD BURK WHO AT COMMERCE WILL BE LEADING THE ECONOMIC STATISTICS ADMINISTRATION AND TRANSITION, THAT GIVES ME A LOT OF COMFORT REALLY. BUT WHEN WE WERE LOOKING TO MAKE SURE THAT WE ARE INSTITUTIONALIZING WHAT WE'RE DOING HERE, YOU HAVE TO START WITH MAKING SURE THERE'S A LINE OF WORK FOR OUR TEAM. BECAUSE THAT DRIVE IT IS DOLLARS WHICH ALLOWS YOU THE HIRE WHICH TOE EXECUTE AND YOU HAVE A CYCLE OF EXCELLENCE THAT CAN DRIVE THIS FORWARD. THAT WAS OUR CORE MISSION FOR THE LAST COUPLE OF MONTHS. AND OUR TEAM THAT INCLUDES JEFF CHEN, JEFF MEISEL, LAURA MCGORMAN, BURTON REIST. WENT AROUND THE DEPARTMENT WITH OUR TEAMS TO SHOW UP WHAT WE CAN DO AND MAKE SURE THERE'S A GOOD UNDERSTANDING OF ALL THE SURFACE SURFACES AND THEN SETTLED TOGETHER AND AGREED WITH OUR PARTNERS ON WHAT WORK WE WOULD BE DOING THE NEXT YEAR. WE THOUGHT IT WOULD BE HELPFUL IF WE SHARE THAT WITH YOU AND GOT INPUT ON IT, HOW ARE WE DOING FOR THE NEXT YEAR AND WHAT'S THE PIPELINE? SO WITH THAT I'LL TURN IT TO DANIEL, JEFF CHEN AND JEFF MEISEL WHO WILL BE COMING UP SOON. >> WE HAVE SOME SLIDES AS WELL. >> IS JEFF MEISEL ON? >> HI, THIS IS JEFF MEISEL. I AM ON. I DON'T KNOW IF YOU CAN HEAR ME. >> THERE HE IS. >> WE HAD HIM FOR A MINUTE. SO TO STAY ON TRACK, JEFF MEISEL, I'M GOING TO HAVE JEFF CHEN START WITH THE SLIDES AND THEN GO BACK TORE YOU, DOES THAT WORK? >> SURE. >> THANK YOU. CAN YOU SHARE THE SCREEN? >> SO IN THIS SESSION WE'LL QUICKLY DESCRIBE EFFORTS, BUT IT MAKES SENSE FIRST TO GIVE A QUICK OVERVIEW OF WHAT WE HAVE DONE IN THE FIRST YEAR AND THEN WHERE WE'RE GOING. SO AS YOU HAVE SEEN FROM TODAY'S PRESENTATION, WE HAVE MADE HEADWAY IN OUR EFFORTS FIRST YEAR WE BROUGHT IN FIVE INAUGURAL BUREAU PARTNERS, WE HAVE SOLICITED AND GOT A LOT OF GREAT ADVICE ON THE CDAC MEMBERS AN SPENT MOST OF THE TIME TRYING TO FIGURE OUT THE SWEET SPOT WHAT IS OUR ROLE WITHIN COMMERCE. THROUGHOUT THIS PROCESS AT THE HEIGHT OUR BUSY SEASON WE ENDED UP WITH 41 PEOPLE INVOLVED WITH THE CDAC EFFORTS IN MANY WAYS, CORE TEAM OF 12, ABOUT NINE PEOPLE WITH HELP FROM ESA, RESIDENCE INCOW BAYTOR OF 15 PEOPLE -- INCUBATOR OF 15 PEOPLE AND FIVE SUMMER ANALYSTS WHO WERE AWESOME AND ENDED UP WITH GREAT PROJECTS. AND MANY OF WHICH WERE TALKED ABOUT TODAY. AND WE'RE CONTINUING SOME OF THIS EFFORT INTO THE NEXT YEAR. SO WHERE ARE WE GOING? WE LEARNED FROM THE FIRST YEAR. OUR SWEET SPOT TURNS OUT AS A COMMERCE RESEARCH AND TECHNOLOGY CONSULTANCY, WE ADVISE BOTH UP TO ALPHA AND WORKING IN CONJUNCTION WITH THIS CIO SHOPS AND DIFFERENT BUREAU WITHIN COMMERCE SO EVERYONE IS ALIGNED MOVING FORWARD IN LOCK STEP SO WE CAN ACHIEVE THE DEPARTMENT'S MISSION. OUR FOCI, WE TRIED TO WORK TO KEY AREAS, HOW WE'RE APPROACHING THE WORK. APPLY DATA INITIATIVES, DATA PLATFORMS MORE INFRASTRUCTURE ORIENTED GROWING OPEN ECOSYSTEM, EDUCATION INCAPACITY, THANKS TO WORK FROM LAURA MCGORMAN AND MR. REIST. WE HAVE MLUs WITHIN COMMERCE, WE'RE SENDING UP STATEMENTS OF WORK WITH EVERY ENGAGEMENT FUNDING BUREAU SO THAT WHEN WE HAVE THIS PROSPECT LIST WE CAN SEE IT GO THROUGH DISCOVERY ON EACH PROJECT SO THAT WE CAN UNDERSTAND THE FEASIBILITY, THE RISKS BENEFITS OF EACH PIECE OF WORK AND WE CAN BUILD AND ADVISE DEPENDING WHAT DISCOVERY TELLS US AND DO RETROSPECTIVES. WE EXPECT UP TO SIX PARTNERS FOR THIS YEAR TWO THOUGH WE HAVE HAD CONVERSATIONS WITH OTHERS VERY RECENTLY LIKE THIS PAST WEEK. THE TEAM WILL SPEND FROM CORE TEAM OF 12 TO 18 MAX. SO IT'S QUITE A BIG CURVE. I'M GOING TO TURN IT OVER BRIEFLY TO JEFF MEISEL TO TAKE US THROUGH THE NEXT TWO SLIDES. >> GOOD AFTERNOON. SO I'LL START WITH THE APPLIED DATA SECTION ARE FOLKS ABLE TO HEAR ME? SO CLOSE. A >> GOOD AFTERNOON, CAN YOU HEAR ME? >> JEFF, YOU'RE ON. WE CAN'T HEAR YOU. >> OKAY. I ONLY DIALED INTO THE AUDIO BUT JEFF CHEN, FEEL FREE THE TAKE OVER. FEEL FREE THE TAKE OVER. -- TO TAKE OVER. I DON'T HAVE CONTROL. NO. (INDISCERNIBLE) >> DO YOU WANT TO PUT YOU UP TO THE SPEAKER, THE MIC, HOLD ON A SECOND. >> CAN YOU HEAR HIM? >> YEAH. JEFF, CAN YOU HEAR ME? >> YEAH. >> ALL RIGHT. JEFF. >> >> SO I WANT TO TALK ABOUT DATA AS OUR FIRST BUCKET OF WORK. THIS IS REALLY EXCITING, WE'RE GOING TO ENUMERATE WHAT THE TECHNOLOGY ALLOWING IF IT DOESN'T ALLOW ME TO PROCEED, JEFF, TAKE OVER AND I'LL SUPPORT YOU BEST I CAN. ON THE CENSUS SIDE OF THINGS YOU HAVE WE'RE WORKING WITH RESEARCH APPLICATIONS IN VA TO UNDERTALK RESEARCH AND DEVELOPMENT FOR END USERS WITH THAT END USER BEING VETERANS. THE GOAL HERE IS WE WANT TO TRY TO GET A PRODUCT THAT MEETS A NEED IN THE FIELD AND UNDERSTAND HOW WE CAN BETTER APPLY TO THE DATA TO SERVE OUR VETERANS. THE SECOND PROJECT, IN THIS BUCKET IS RAPID PROTOTYPING. SO WE WANT TO BUILD SMALL PILOTS AND RECOMMENDATION ENGINE TO SUPPORT A BUILD OUT OF OUR FUTURE GENERATION DISSEMINATION PLATFORM WHICH IS CALLED -- SO WHAT THE COMMERCE DID IS ALLOWK (INDISCERNIBLE) OUR TEAM TO ACCELERATE THE UNDERSTANDING OF NEW TECHNOLOGIES, NEW APPROACHES THAT THEY CAN THEN EXPERIMENT WITH AS BUILDING OUT. >> CENSUS AN NON-CENSUS FOR NIST WE HAVE A NUMBER OF PROJECTS WITH OUR PARTNERS THERE INCLUDING SOME EFFORTS ON -- TO BETTER UNDERSTAND URBAN INTERFACE DATA THAT GETS AT WILD FIRE PROBLEM, NIST HAS A RESEARCH PROGRAM FOCUSED ON UNDERSTANDING THE WILD FIRE VULNERABILITIES SO WE UNDERSTANDING HOW TO COLLECT DATA, PROVIDE RECOMMENDATIONS HOW TO IMPROVE SPATIAL DATA COLLECTION SO THEY CAN APPLY MISSION LEARNING ALGORITHMS TO UNDERSTAND VOLNERIBILITYS. AN EXTENSION TO ONE OF THE PROJECTS FROM THIS YEAR WE HAVE ALSO WE WILL BE DEPLOYING A CLIENT -- IDENTIFICATION ALGORITHM WITH NIST HOLLINGS MANUFACTURING EXTENSION PARTNERSHIP PROGRAM TO REACH TOUT TO COMPANIES THAT MAY BENEFIT FROM MEPs CLIENT SERVICES AND THAT'S USING SOME WORK WE DEVELOPED IN OUR FIRST YEAR. >> SO UNDER THE ECOSYSTEM WE'RE GOING TO FOCUS ON THE USE AND IMPACT OF OPEN DATA AND SO I'LL BEGIN WITH SYSTEMS. THE FIRST PROJECT QUITE EXCITING IS WITH WIKIPEDIA. SO WHAT WE'RE LOOKING TO DO IS AUTOMATE HOW WE POPULATE CENSUS DATA INTO WIKIPEDIA PAGES. THE REASON THIS IS SO IMPORTANT IS PRIMARILY USERS GET DATA FROM CENSUS BY EITHER GOOGLING IT OR GOING TO WIKIPEDIA. SO THIS WILL BE A WAY THAT WE CAN ENSURE OUR DATA IN THE LATEST DATA AVAILABLE FOR WHERE OUR USERS GET IT. THE SECOND PROJECT IS CONTINUATION AROUND OUR SOFTWARE DEVELOPMENT KIT WHICH SUBPOENA DEVELOPER OUTREACH THAT ALLOWS US TO PARTNER WITH PRIVATE SECTOR ORGANIZATIONS, UNIVERSITIES, GROUPS LIKE (INAUDIBLE) AMERICA AND GOING TO COMMUNITIES TO EM H THEM SOLVE PROBLEMS BY GETTING WHAT'S USING DATA FROM CENSUS FROM COMMERCE, AND FROM THE -- MORE BROADLY. >> GOING BACK TO NIST WE WILL BE WORKING WITH THE MATERIAL, THE NML LAB AT NIST FOCUSING ON IMPACT QUANTIFICATION OF STANDARD REFERENCE DATA. THIS DATA HAS -- HAS LONG STANDING HISTORY OF CONTRIBUTING TO A VARIETY OF FIELDS OF RESEARCH AND SO THERE'S A PARTICULAR INTEREST IN UNDERSTANDING HOW DO THAT DATA PERFORATE THROUGH DOWNSTREAM USERS ECOSYSTEM INCLUDING FROM THE PATENT SIDE TO THE FINANCIAL SIDE. HOW THAT DATA IMPACTS BUSINESSES AND RESEARCHERS AROUND THE COUNTRY IF NOT THE WORLD. WITH PTO, WE ARE DIGGING IN A BIT MORE ON TO THE STRATEGY SIDE INITIALLY WITH UNDERSTANDING THEIR OPEN DATA USER STRATEGIES. THERE IS AN OVERARCHING NEED TO UNDERSTAND WHAT ARE THE -- WHAT ARE THE ACTUAL NEEDS OF PATENT DATA USERS AND HOW WE -- HOW PATENT -- PTO CAN BE MORE USER DRIVEN IN THEIR APPROACHES. FROM DATA PLATFORM SIDE WE'RE FOCUSING ON FIGURING HOW CAN THE SYSTEMS BETTER PROVIDE INSIGHT AND UTILITY TO USERS WHETHER ON DEVELOPER SIDE OR ACTUAL DIRECT TO USER PROJECT. SO WITH THE BUREAU OF ECONOMIC ANALYSIS WE ARE CONTINUING REDESIGN OF THEIR RIMS PROJECT REGIONAL INPUT OUTPUT DATA AND WE'RE TRYING TO IMPROVE THE USER EXPERIENCE OF GETTING THE RIMS MULTIPLY DATA. WITH PTO WE ARE DIGGING IN A BIT MORE WITH THEIR API STANDARDS WORK. THEY HAVE SOME GREAT DOCUMENTATION COMING OUT NEXT MONTH OR SO AND WE'RE GOING TO WORK WITH THEM TO PERHAPS IMPLEMENT SOME REFERENCE API IMPLEMENTATIONS. >> THEN ON THEOGRAPHY PRODUCTS -- THE GEOGRAPHY PRODUCTS FOR CENSUS, ONE PROBLEM WE NEED THE ADDRESS IS HOW MANY USERS ARE UNAWARE OF GEOGRAPHIC COVERAGE OF CENSUS PRODUCTS SO WHAT WE ARE LOOKING TO DO IS IMPROVE UPON THE USER EXPERIENCE SO THAT WE CAN PROVIDE VALUE TO USERS AND THEY CAN SEE THE FULL BREADTH OF OUR CAPABILITIES . VECTOR ASSESSMENT BUT AS YOU KNOW IT OPERATES SEVERAL PROJECT PROJECT, COLLECTION PURPOSES, HOWEVER WITH ADVENT OF VECTOR TRIAL TECHNOLOGY WITH USER BEHAVIOR MOVING MORE MOBILE, IT'S IMPORTANT WE UNDERSTAND HOW WE CAN UTILIZE VECTOR TRIALS IN MANY PRODUCTS, SO THAT WILL BE ANOTHER AREA COMMERCE DATA SERVICE WILL BE INVESTIGATED. >> AND LASTLY, OUR EDUCATION AND CAPACITY BUILDING EFFORTS, ARE ONGOING, OUR SUCCESSFUL COMMERCE DATA ACADEMY WORK WILL BE GOING INTO SECOND YEAR. WE WILL BE LOOKING TO DO ANOTHER ROUND OF THE RESIDENCE PROGRAM WHICH HAS A LOT OF -- GRAPEFRUIT WITH CENSUS, I ONLY SERVING ON A COMPETING TASK FORCE TO HELP INFORM THEIR INTERNAL METADATA AND DATA DISSEMINATION AND DATA MATCHING AND MERGING. MERGING CAPABILITIES. AND THAT REALLY LOOKING FOR THE BUREAU'S LONG TERM RESEARCH ACTIVITIES. LASTLY WE'RE GOING TO CONTINUE TO WORK ON THIS DATA USABILITY PROJECT WHICH RECENTLY WON A FETES GROUP AWARD. WE HAVE A COUPLE OF OTHER NEW TUTORIALS THAT WILL BE GETTING OUT THE NEXT COUPLE OF WEEKS. >> SO THAT'S ESSENTIALLY A SELECTION OF WHAT WE'RE LOOK AT FOR THIS YEAR. WE'RE GOING TO DO DISCOVERY ON ALL OF THEM, NOT ALL TAKE THE SAME TIME BUT WE'RE GOING TO TAKE A VERY SYSTEMATIC APPROACH TO TACKLING THESE PROJECTS AND MAKING SURE WE DO THEM RIGHT. >> I'M ACTUALLY THE FACILITATOR SO THAT MEANS I MAYBE WILL GO FIRST ON COUPLE OF QUESTIONS. THIS REALLY AN OPEN DISCUSSION. I GUESS FIRST, ONE REACTION TO THIS, GREAT WORK AHEAD, THINK WE'RE LOOKING FOR TO IT. ONE THING I DIDN'T SEE UP THERE THAT HAS COME UP IN OUR PAST DISCUSSIONS ESPECIALLY SUMMER SESSION WAS MARKETING THESE PROJECTS AND WHEN I SAW THE PROCESS THAT YOU HAVE FOR GOING THROUGH THE PROTECT SOMEWHERES I DIDN'T SEE THAT THERE, WOUNDER IF YOU CAN TALK ABOUT IF YOU ARE THINKING ABOUT THAT OR INTEGRATING THAT MORE INTO YEAR 2 NOW YOU'RE GETTING MORE INSTITUTIONALIZED, THAT'S ONE POINT. A SECOND THAT I WANT TO RAISE, WAS AROUND WIKI DATA, WIKIPEDIA DATA AND WHAT YOU'RE THINKING THERE IN TERMS OF THE USER EXPERIENCE AND VERIFIABILITY. AS A RESEARCHER, YOUR -- I ALWAYS TELL MY STAFF, DON'T USE WIKIPEDIA BECAUSE YOU CAN'T TRUST IT SO IF YOUR GOAL IS TO GET THAT DATA THERE, IS YOUR GOAL TO GET IT THERE SO IT'S ACCURATE OR BECAUSE IT'S TRUSTED SOURCE? IF IT'S TRUSTED SOURCE HOW ARE YOU THINKING ABOUT THAT, CAN THERE BE A SEAL OR SOME EVIDENCE THAT IT'S ACTUALLY THE CORRECT DATA, YOU HAVE ANY THOUGHTS ON THAT, THAT WOULD BE USEFUL. THIRD IS ON YOU MENTION NUMBER OF INTERESTING PROTECT SOMEWHERES LIKE THE VETERANS DATA PROJECT AND WILD FIVE PROJECT. TO WHAT EXTENT IS THIS CROSS AGENCY WORK? OUTSIDE THE DEPARTMENT OF COMMERCE? IT IS OBVIOUSLY IN MANY CASES YOU'RE TALKING DHS, VA, HOW IS YOUR INTERFACING WITH THEM GOING? >> SOUNDS GOOD. I WILL TAKE THE FIRST ONE THEN I WILL -- JEFF AND I WILL -- JEFF MEISEL AND I WILL JUMP BACK AND FORTH WITH THE PHONE TO THE MIC. SO IN TERMS OF MARKETING WE I THINK IT CAN BE DIVIDED TO INTERNAL AND EXTERNAL. INTERNALLY WE ARE SETTING UP STANDARD PROCEDURES TO CONSTANTLY PUSH UP GOOD NEWS INTERNALLY. MEETING WITH BUREAU LEADERSHIP ON THEIR -- ON A REGULAR BASIS, GET THROUGH DATA ACADEMY WE HAVE A GREAT WAY OF DOING CONDUCTING OUTREACH, THAT'S ONE OF THE EASIEST WAYS TO BRING NEW SUPPORT INTO THE CDS FAMILY. EXTERNALLY WE ARE CONNECTED TO THE BUREAU OF FEDERAL FAMILY OF DATA ORGANIZATIONS AS WELL AS WE ALSO SPEAK QUITE A BIT EXTERNALLY ON THE STUFF WE DO INTERNALLY. OF COURSE, ENGAGING WITH THE CDAC MEMBERS, THAT'S A NO BRAINER. AS FOR WIKI DATA, I'LL PUT UP TO JEFF FIRST THEN I'LL JUMP IN. >> THANK YOU, SO ONE OF THE THINGS -- ONE OF THE FIRST AREAS WE THINK WE HAVE EAR HANGING FRUIT IS TO ENTER THE DATA THAT'S ALREADY ON WIKIPEDIA IS THE LATEST AND MOST RELEVANT DATA SET BEING REFERENCED. THEY HAVE A PROCESS THAT GOES THROUGH APPROVALS GROUP THAT ALLOWS FOR AUTOMATED CHANGES. SO AS WE GET INTO THE DISCOVERY OF THIS PROJECT, WE'LL LOOK TO WORK CLOSELY WITH THEIR TEAM, TO UNDERSTAND BEST PRACTICES HOW FOLKS ALREADY -- AND MOST CERTAINLY DEVELOP A SMALL SUBSET OF PROBLEMS TO TACKLE FIRST. BUT REALLY WE WANT TO ENSURE WE'RE GETTING OUR DATA OUT TO THE PUBLIC AND IT'S THE MOST RELEVANT UP TO DATE AND CORRECT DATA FOR WHAT THEY'RE LOOKING FOR. SO THAT'S WHAT I WOULD ADD ON THE WIKI DATA PROJECT. >> FOR A MOMENT ON THE EXTERNAL MARKETING WHICH WAS A VERY GOOD QUESTION. WE OBVIOUSLY FOCUS QUITE A BIT ON THE -- TO MAKE SURE WE GET THE WORD OUT IN A NUMBER OF CHANNELS. BURTON LUCKILY -- OUR ACCOUNTS OFFICEND AWE WHOLE TEAM ON EVERY SINGLE PRODUCT WE HAVE, A RELEASE, WE BROUGHT IN EXPERTS LIKE TERI OLNESKY FROM SILICON VALLEY TO TEACH US HOW DO YOU MAKE SURE YOU PRODUCTIZE, TRAIN, THE TRAINING SESSIONS, THE RELEASES, ALPHA, BETA. ALL THE THINGS ALONG THE CHAIN WE NEEDED TO BE DOING ARE NOW PART OF OUR KNOWLEDGE BASE. OUR PRESS TEAM HERE AT ESA WILL BE HERE TO SUPPORT THE COMMERCE DATA SERVICE OF COURSE LIKE WE SERVE OTHER AGENCIES IS AND THE GOAL IS TO ELEVATE AS MUCH AS WE CAN UP THE CHAIN NOW AND IN TO THE NEW ADMINISTRATION. JEFF REALLY HAS DONE A TERRIFIC JOB OF BEING A THOUGHT LEADER AND A SPEAKER OUT THERE, BUT THAT RESULTS IN A LOT OF ELEVATION AND AMPLIFICATION. I HAVE TO TELL YOU CANDIDLY, WE NEED HELP. WE ASK FOR THIS EVERY TIME BUT YOU WOULD BE SHOCKED HOW MUCH ONE TWEET, ONE REPETITION GETTING IT FROM YOUR COMPANIES AND FROM -- THE KNOWLEDGE -- THE CONNECTIONS YOU HAVE, HOW MUCH IT MATTERS. LITERALLY ONE OR TWO THINGS FROM Y'ALL AND FROM OTHER AMPLY FIRS IN THIS AREA WHO HAVE FOLLOWED US, IT DRIVES APPLICATIONS TO US, USAGE OF DATA, DRIVES USAGE OF PRODUCTS. IF IT HELPS YOU FOR US TO SHOW YOU HOW MUCH YOU'RE DOING WHEN YOU DO THAT, WE WOULD BE HAPPY TO DO IT. WE KNOW AND CAN SHOW YOU. SO ANYTHING YOU CAN DO WOULD BE HELPFUL BUT WE ALSO TAKE ADVICE WHERE WE NEED TO BE SPEAKING, TALKING AND ENGAGING WITH THE PRESS BECAUSE WE ARE PROUD OF WHAT WE'RE DOING AND WE NEED THE FIND BETTER WAYS TO SHARE. >> ABSOLUTELY. AS FOR THE LAST QUESTION ABOUT CROSS AGENCY WORK. WE -- SO FOR THE VETERANS PROJECT WE ARE ENGAGING THE VA. SOME STATE LOCAL VETERANS ORGANIZES. WE ARE TAKING PREVIOUS INPUT THAT PROVIDES BSOs GREAT WORK THAT AUSTIN CHAMPIONED IN THE PAST AND WHERE POSSIBLE WE ARE REACHING OUT AND GETTING INPUT INTO HOW TO IMPROVE IT SO NOT REINVENTING THE WHEEL. HAD A GREAT DISCUSSION WITH BILL BEFORE ABOUT MAKING SURE WE CONNECT WITH THE RIGHT PEOPLE, SO THAT WE CAN -- WE'RE STAND ORGANIZE ON THE SHOULDERS OF GIANTS AND DELIVERING BIGGER AND BETTER THINGS. >> I'M GOING TO ASK TO GO TO COTI BECAUSE I THINK WE SKIP HER THE LAST ROUND. >> FABULOUS JOB, THANKS SO MUCH FOR DOING THIS. I THINK IT'S VERY GOOD TO SEE HOW MANY TOOLS YOU BROUGHT INTOING YOUR ORGANIZATION AND HOW YOU BROUGHT DWIGHT DIFFERENT WAYS OF DO -- QUITE DIFFERENT WAYS OF DOING THINGS TO THE DEPARTMENT OF COMMERCE. IN TERMS OF TEACHING THE WORLD HOW TO DO THIS, I MENTIONED BEFORE THAT THE CONFERENCE THAT BRINGS 5,000 TOGETHER FROM INDUSTRY BUT ALSO MOSTLY GOVERNMENT, ACADEMIA, THOSE WHICH CARE ABOUT SCIENCE AND HOW IT CAN BE USED EFFECTIVELY, THINK THINK THAT WOULD BE GREAT VENUE TO PROPOSE A SESSION ON HOW TO MAKE DATA MORE OPENLY AVAILABLE TO INDUSTRY AND ACADEMIA, ACROSS THE AGENCIES. ONE COMMENT I HAD TO DANAH'S PRESENTATION WAS DATA VISUALIZATION LITERACY, I BELIEVE IN TODAY'S TIME YOU HAVE TO FIND WAY PEOPLE DO NOT ONLY READ AND WRITE TEXT BUT ALSO READ AND WRITE DATA. HOW DO YOU GET THERE? IT'S NOT DONE BY JUST ADDING ONE SCATTER PLOT AND OTHER DATA VISUALIZATION TO CURRICULUM, YOU NEED TO CATCH THOSE THAT ARE ALREADY OUT OF THE EDUCATIONAL FORM, YOU HAVE TO GO INTO LIBRARIES AND INTO SCIENCE MUSEUMS AND THERE COULD BE ALL KINDS OF PARTNERSHIPS THAT MAKES VERY RICH DATA AVAILABLE IN TWO SCIENCE MUSEUMS AND I'M HAPPY TO TALK MORE ABOUT THAT. Q. I AGREE, THAT WOULD BE HELPFUL HONESTLY. >> JUST A QUICK QUESTION ABOUT -- THE GREATEST OF PROJECTS WHAT ARE YOUR CRITERIA FOR PRIORITIZING THE ONES YOU CHOOSE TO DO IN 2017? >> WE ENDED UP WORKING WITH PROJECT CHAMPIONS WITHIN EACH BUREAU TO UNDERSTAND WHAT ARE THEIR NEEDS SO THEY PROVIDE US INITIAL OVERALL LIST OF POTENTIAL PROJECTS. FROM THAT WE TRY TO PRIORITIZE BASED ON WHETHER OR NOT THERE'S A CLEAR CHAMPION WHAT ARE THE POTENTIAL RISKS INVOLVED AND WHAT ARE THE -- IF THERE'S A CLEAR UNDERSTANDING WHERE WE'RE GOING WITH IT. IF IT'S MORE A CONSULTING OR ADVISORY OR BUILD PROJECT, THAT ALL FACTORS IN. BUILD, IT'S THE PROOF IN THE PUDDING SO WE HAVE TO END UP DOING DISCOVERY ON EVERYTHING. THAT WAS BROUGHT UP PROVIDED BY CENSUS, A NUMBER WILL BE DEFINITELY GET THE LION'S SHARE OF THE EFFORT. BUT WE WILL DO -- LOOK AT ALL OF THEM. >> SO THE REASON I ASK, TRYING TO UNDERSTAND THE BALANCE BETWEEN HOW MUCH OF THIS ABOUT BUILDING UP THE SORT OF DATA SCIENCE MUSCLE WITHIN COMMERCE AND HOW MUCH IS ABOUT SOLVING SPECIFIC PROBLEMS FOR SPECIFIC AGENCIES. AND BASICALLY HOW YOU'RE TRADING OFF BETWEEN TWO. YOU DESCRIBED EARLIER IN THE DAY, YOU'RE GOING TO DESCRIBE 2017 AS AN INVESTMENT YEAR, KIND OF LIKE -- I FORGET THE WORD YOU USED BUT WHAT THE BALANCE BETWEEN THOSE TWO WAS. >> OKAY. WELL, THERE IS SPECIFIC PROBLEMS, SOME MORE BETTER DEFINED THAN OTHERS. SO CHRIS AND JEFF MEISEL TALKED RAPID PROTOTYPING, THE GENERAL NEED TO GO TO PLATFORM FORMAT AND CENSUS AND THAT MAY TAKE A FORM IN THESE THREE OR FOUR POTENTIAL, NOT PREVIOUSLY TESTED AREAS. SO WE WOULD BE DOING THAT SORT OF WORK AND THAT'S TOWARD -- THAT PUSHES TOWARDS BETTER PLATFORM. OTHER CASES, A VERY SPECIFIC LOOKING AT WRITTEN INTERFACE, LIKE HOW DO YOU IMPROVE DATA COLLECTION BETTER. AND THAT'S VERY MUCH A VERY, VERY SPECIFIC TARGETED PROBLEM THAT WE'RE ADDRESSING. I THINK BECAUSE WE'RE EXPANDING THE OVERALL PORTFOLIO, THEN BY VIRTUE YOU GROW AT THE SAME TIME >> >> STAN LAST QUESTION, COMMENT? >> ONE THING I GUESS LOOK THROUGH YOUR -- THE PRIORITIES FOR 2017, SOMETHING THAT OCCURRED -- JURIED OUT WAS -- JUMPED OUT, OBVIOUSLY IT'S TECH FOCUSED AS WELL AS IT SHOULD BE BECAUSE A LOT IS ABOUT TECH ENABLEMENT AND GOT PROJECTS THAT YOU'RE HELPING BASICALLY BE GOOD TECHNOLOGY, AS YOU SAID AT LUNCH TECHNOLOGY CONSULTANT INTERNALLY AND KIND OF A REPOSITORY FOR BEST PRACTICES, YOU ALSO HAVE GOT THE DATA ACADEMY PIECE WHICH IS LIKE BUILDING UP THE CAPABILITY TEACHING PEOPLE THEMSELVES, THERE ARE SOME PROBLEMS THAT ARE META PROBLEMS THAT ARE LITTLE BIT LESS TECHNOLOGY FOCUSED NOT SURE WHERE THEY LIE IF ONE THE DATA SERVICE RESPONSIBILITY BUT THINGS WE TALKED ABOUT ARE THINGS LIKE HOW TO DO WHAT LESS IS LONG TAIL OF USER ENGAGEMENT SO THE FACT THAT A LOT OF PEOPLE MAYBE WANT SOME THINGS OUT OF COMMERCE, THEY DON'T KNOW HOW TO INTERACT AND COWE LESS AND CREATE COMMUNE -- COALESCE AND HAVE A BETTER DIALOGUE, THAT'S A SOCIOLOGICAL ETHNOGRAMMIC NOT A TECHNICAL PROBLEM BUT IT DOES FACILITATE THE TECH TRANSFER IF YOU HAVE A BETTER FORM OF ENGAGEMENT WITH THE COMMUNITY. THE OTHER THING IS WE DO OFTENTIMES TALK ABOUT DISCOVERY AND CONNECTION PROBLEMS. WHICH IS DATA.GOV WHICH I THINK IS -- IT'S BETTER THAN WHAT IT WAS BEFORE AND SOMETIMES FEEL LIKE WE KICK IT AND SO IT COULD BE BETTER, BUT THE REALITY IS IT'S BEAR, AT LEAST ONE PLACE PEOPLE CAN GO TO FIND STUFF, BUT A LOT OF PROBLEMS WE TALK ABOUT IS HOW DO FOLKS HOW WE IMPROVE THAT ABILITY FOR RANDOM COMMUNITIES OF PEOPLE TO SAY I WONDER WHERE TO GET DATA ON THIS, FINE IT AND THEN IN THAT SAME INTERFACE FIND OTHER CONNECTIONS OF WHERE PEOPLE USE THAT SAME TYPE OF DATA AND CODE FOR IT AND EVERYTHING ELSE. SO IN THE NECESSARILY A TECH -- TECHNICAL PROBLEM IT'S AN INFORMATION SYSTEM PROBLEM. BUT CDS, IF THAT'S IN THE REMIT OR NOT. >> WE ARE CERTAINLY LEARNING A LOT THOUSAND ENGAGE PUBLIC THROUGH DIFFERENT CHANNELS, MORE THAN OTHERS. THERE'S ACTUALLY A DATA OF WORK THAT GETS MORE TOWARDS THE PROJECTS THAT'S PUSHING ON DATA EQUALITY. AND ESSENTIALLY THERE ARE THOSE CHANNELS ENGAGING WITH COMPANIES AND FIGURING WAYS TO ENGAGE WITH THE PUBLIC DIRECTLY SO THAT NETWORKS AND INNOVATORS CAN PROVIDE THEIR INPUT TO DIFFERENT PROBLEMS. AND THINK THROUGH DIFFERENT WAYS OF IMPROVING THAT CONNECTIVITY DIRECTLY WITH THE PUBLIC. THAT'S KIND OF LIKE IN THE WORKS. LAURA MCGORMAN MIGHT SPEAK TO THAT LATER. AND IT'S -- >> 2017? >> IT'S ONGOING. >> SO LET'S -- THANKS JEFF AND EVERYONE, LET'S TAKE A SHORT BREAK. WE'RE A LITTLE BEHIND SCHEDULE BUT NOT AS BAD AS WE HAVE BEEN IN THE PAST. SO THAT'S A GOOD THING. TAKE A SHORT BREAK AND RECONVENE IN ABOUT TEN MINUTES. >> THANKS TO JEFF MEISEL ON THE PHONE TOO. THANKS, JEFF. >> HELLO. I DON'T KNOW IF YOU CAN HEAR ME BUT I CAN'T HEAR. HELLO. IT LOOKS LIKE YOU CAN HEAR ME BUT I CAN'T HEAR ANYTHING ON YOUR. I COULD HEAR EARLIER. CAN YOU GIVE ME A THUMBS UP IF YOU CAN HEAR ME? HELLO. >> I CAN HEAR NOW. I LOST YOU AGAIN. TEST TEST TEST TEST TEST. I CAN'T HEAR YOU. TEST TEST TEST TEST TEST TEST TEST. >> OKAY. I CAN HEAR YOU. >> AND LISA WILL ALSO JOIN US, WE (INDISCERNIBLE). >> THANK YOU. I WE'RE ALL PART OF (INDISCERNIBLE) UPSTREAM TALKING (INDISCERNIBLE) ABOUT HOW TO DEMOCRATIZE (INDISCERNIBLE) COMMON GOAL (INDISCERNIBLE) >> WHEN YOU THINK ABOUT ADVANCING DATA QUALITY (INDISCERNIBLE) WHEN YOU THINK ABOUT MAKING ALL THE DATA AVAILABLE ON (INDISCERNIBLE) ESSENTIALLY -- VERY DIFFERENT (INDISCERNIBLE) (INDISCERNIBLE) TALKING ABOUT MAYBE THE DATA OPEN (INDISCERNIBLE) THERE ARE TODAY (INDISCERNIBLE) PRETTY CLEAR OVERVIEW OF WHAT THE (INDISCERNIBLE) (INDISCERNIBLE) SO I WANT TO START WITH A BRIEF OVERVIEW THE FIRST IS HOW MANY (INDISCERNIBLE) BUT IF YOU THINK ABOUT (INDISCERNIBLE) (INDISCERNIBLE) 53%. SOFT UNDERSTANDING THE (INDISCERNIBLE) THIS ISN'T (INDISCERNIBLE) FACT FINDER LOCAL GOVERNMENT LARGE AMOUNTS OF DATA (INDISCERNIBLE) COMPANIES, PRIVATE INDUSTRY CENSUS DATA. (INDISCERNIBLE) (INDISCERNIBLE) (INDISCERNIBLE) ADVISORY SERVICES (INDISCERNIBLE) (INDISCERNIBLE) (INDISCERNIBLE) INVOLVED IN OUR COMMUNITIES (INDISCERNIBLE) DEVELOPING A STRATEGIC PLAN (INDISCERNIBLE) ADVANCE MARKETING TOOLS THAT YOU HAVE TO PAY FOR TO DO SOME (INDISCERNIBLE) WALK AWAY. (INDISCERNIBLE) ACCESS TO DEMOGRAPHIC DATA (INDISCERNIBLE) SPECIFIC (INDISCERNIBLE) THE ACTUALLY (INDISCERNIBLE) WE NEED HELP WITH (INDISCERNIBLE) PARTNERS IN THE PRIVATE SECTOR (INDISCERNIBLE) SO WE HAVE BEEN ASKED BY (INDISCERNIBLE) AND I THINK SUCCESS -- (INDISCERNIBLE) MORE NON-PROFIT NEXT STEP. AND (INAUDIBLE) PROVIDE THE BEST SERVICES POSSIBLE (INDISCERNIBLE) YOUR ABILITY TO SAY THANK YOU VERY MUCH FOR OFFERING (INDISCERNIBLE) THIS IS THE BEGINNING OF A WHOLE LIFE CYCLE ¶¶ (INDISCERNIBLE) PREFERRED DISCUSSION AT THE END. (INDISCERNIBLE) >> VERY THOUGHT PROVOKING. I KNOW WE WANT TO GET TO QUESTIONS. I WANT TO MAKE SURE (INDISCERNIBLE) THEN I DO WANT TO HIGHLIGHT SOME OF THE COMPANIES (INDISCERNIBLE) TRY TO HELP STEVE AND OTHERS, SO I WILL ASK TO GO TO FOUNDER AND >> THANK YOU, GOOD AFTERNOON. I WANT TO TAKE YOU ON A SHORT JOURNEY OF OPEN DATA INITIATIVES THAT HAVE BEEN GOING ON AT THE CENSUS BUREAU AND VERY CLOSELY TIED IN AS LAURA WAS SAYING FROM PROBLEMS, SO THE CENSUS DATA API LAUNCHED IN AUGUST OF 2012 AFTER BOTH CLOSED AND OPEN BETA TEST TESTING, USER FEEDBACK WAS IMPORTANT FROM DAY ONE. OUR PROBLEM WAS NOT DO WE HAVE DATA FOR API OUR PROBLEM WAS WHAT OF OUR HUNDREDS OF DATA SETS SHOULD WE START WITH IN THE API. AND WE LAUNCHED WITH THE DECENNIAL CENSUS AND ACS FIVE YEAR DATA. THERE'S NOW DOZENS OF DATA SETS AVAILABLE THROUGH THE API FROM SEVERAL INSTANCES OF THE ECONOMIC CENSUS QUARTERLY WORK FORCE INDICATORS, WAS ALSO MENTIONED THE CENSUS BUREAU IS A ME SRI USER -- HEAVY USER OF ITS API, IT DRIVES APPLICATIONS OF OUR POPULATION CLOCK, AND MY CONGRESSIONAL DISTRICT CREATED WIDGETS OFF THE API SO FOR EXAMPLE WHEN MEMBERS OF CONGRESS EMBED MY CONGRESSIONAL DISTRICT ON THEIR HOUSE SITES, IT'S UPDATED AS SOON AS THAT DATA IS RELEASED. IT WAS UPDATED FOR EXAMPLE SEPTEMBER WHEN THAT DATA SET WAS REFRESHED. SO IT'S REALLY ENABLED US THROUGH THE API AND THESE WIDGETS TO GET DATA FAR BEYOND CENSUS.GOV. IT'S ALSO REDUCED DEVELOPMENT TIME IN HOUSE BEING ABLE TO USE THE API. WE POWER FEATURES ON OUR WEBSITE SUCH AS OUR SMART SEARCH FEATURE WHICH PULLS DATA FOR QUERIES RIGHT WITHIN OUR SEARCH ENGINE. FEEDBACK AS I MENTION DURING BETA TESTING AND POST LAUNCH HELPED US EVOLVE FEATURES AND FUNCTIONALTIVE WITHIN THE API. LAUNCHING THE ARCPI IS REALLY -- API IS THE FIRST STEP IN THAT JOURNEY. WE GOT A LOT OF POSITIVE FEEDBACK, THANK YOU FOR GETTING THIS DATA OUT, MACHINE READABLE FORMATS WE CAN USE IT BUT THERE WERE DEFINITELY USABILITY ISSUES WITH THE API, WITH THE DATA ITSELF. AND ESPECIALLY AROUND MAPPING. IN TALKING WITH DEVELOPERS WE HEARD I'M SPENDING HOURS CONVERTING DIFFERENT GEOGRAPHY GEOGRAPHIES, DEALING WITH SHAPE FILES IS REALLY DIFFICULT. THIS WAS SOME FEEDBACK ACTUALLY FROM A HACK ONTHON WITH ZILLO. I TALK MORE ABOUT HACKATHONS IN COUPLE OF MINUTES, SO IT WAS REALLY HARD TO USE API IN CONJUNCTION WITH SPATIAL DATA TO USE MAPPING, WE HAVE SEEN SO MANY TIMES TODAY HOW IMPORTANT MAPPING IS TO BE ABLE TO VISUALIZE, VISUALIZE DATA. THIS IS IN ADDITION TO COMPLEXITY OF CENSUS DATA ITSELF WHICH UNFORTUNATELY CAN BE VERY OVERWHELMING. SO WHAT WE DID IS WE TOOK THIS FEEDBACK AND WORKED ON THE CITY STK PROJECT THROUGH AGILE DEVELOPMENT, WE CAN PULL TOGETHER WE GET FEEDBACK AND ENGAGE AT HACKATHONS, WE'RE IN THE NATIONAL DAY OF HACKING, WE DID ONE WITH AMAZON THEIR TEAM RECENTLY, WE DID ONE A FEW WEEKS AGO IN PUERTO RICO WHICH HELPED IDENTIFY ISSUES WITH THE PUERTO RICO DATA SO HACKATHONS, I'LL WRAP UP QUICKLY, OUR THE ARE THE BEST VENUE FOR LIVE VISIBILITY TESTING AROUND ARCPI BECAUSE THE DATA ITSELF IS SO DIFFICULT TO USE. >> SO I'LL BE BRIEF. I THINK LAURA AND LISA HAVE DONE A GREAT JOB DEFINING THE PROBLEM, WE'RE PASSIONATE ABOUT OPENING DATA AND HAVING DATA BE A RESOURCE FOR EVERYONE TO USE TO CREATE MORE ECONOMIC GOODNESS, SOLVE SOCIAL PROBLEMS, ET CETERA. WE HAVE CERTAINLY MADE EFFORTS TO TRY TO MAKE THAT HAPPEN BUT WE'RE ONLY AT THE BEGINNING OF THE JOURNEY SO WE HAVE DONE THINGS LIKE HACKATHONS, MARKETING, WE'RE ENGAGING WITH OPEN SOURCE COMMUNITIES, DATA COMMUNITIES WITH DATA USERS AND ANALYSTS WE'RE SPEAKING PUBLICLY, ET CETERA, ET CETERA. BUT I WOULD ARGUE THAT BECAUSE WE'RE EARLY STAGE AND JUST STARTING TO RAMP NOW THE NEXT LEVEL OF COMMUNICATION AND MARKETING AROUND THIS OPEN DATA EFFORT NEED TO BE BIGGER THAN INDIVIDUALLY OR COLLECTIVE GROUP OF 3 TO 5 PEOPLE IN THE COMMERCE DATA ACADEMY. ANY SUCCESSFUL PRODUCT AT THE BEGINNING STAYS CLOSE TO THE FOUNDING GROUP AND YOU LET IT OUT A LITTLE BIT BUT THE ADOPTION, MOMENTUM AND THE ENCOURAGEMENT WE HAVE SEEN FROM ANY GROUP PEEKING TO IS -- SPEAKING TO IS PHENOMENAL WHICH GIVES ME THE INTUITION THIS IS A BIGGER MESSAGE TO THE AMERICAN PUBLIC. SO ESSENTIALLY THE POINT I'M GOING O IF MAKE IS THERE'S STILL LOTS TO DO, THERE'S LOTS TO COMMUNICATE, THE BEGINNING EFFORTS WE HAVE MADE HAVE BEEN FANTASTIC. IF COMMERCE AND GOVERNMENT WANT OPEN DATA TO BECOME A THING IN AMERICA AND IF WE WANT TO USE COMMERCE DATA AS WELL AS DATA FROM ANY OTHER FEDERAL AGENCY TO BE A CATALYST FOR SOCIAL CHANGE OR ECONOMIC GOOD WE NEED A BIGGER PUBLIC AWARENESS PROGRAM AND OUTREACH AND EDUCATION EFFORT AROUND IT. THAT MEANS EVERYTHING FROM LIKE MAYBE WE HAVE THE PEOPLE'S CLUB, WHERE ALL THE BEST DATA THAT CAN BE OPEN IS PUT. THEN WE HAVE COMPANIES OR OTHER STARTUPS OR ENTREPRENEURS WHETHER THEY'RE SOCIAL OR PRIVATE, WORK OFF THAT CLOUD TO DEVELOP TOOLS AND THINGS THAT ANSWER, SOLVE PROBLEMS OR CREATE THE ECONOMIC GOODNESS THAT WE'RE PROPOSING. AGAIN, I THINK THIS IS SOMETHING THAT CAN SCALE AND BE BENEFICIAL ON A MUCH WIDER PLATFORM THAN WHAT WE'RE DOING NOW BUT AGAIN, IT'S TECHNOLOGY, IT'S MARKET EDUCATION, IT'S DIFFERENT KINDS OF FUNDING, ET CETERA ET CETERA. SO WE HAVE STARTED SOMETHING, THERE'S MOCKER WORK TO BE DONE. -- MORE WORK TO BE DONE. >> I'M LOOKING FORWARD TO THE DISCUSSION. LAURA, IF YOU WOULDN'T MIND GOING BACK TO THE SLIDE THAT HAVE COMPANIES THAT STEP FORWARD. I WANT TO SAY A WORD OR TWO BEFORE I TURN IT OVER TO ANTHONY GOLD BLOOM AND KEVIN FROM SOCRADA. I'M TRYING TO SPUR IDEAS FOR NOT ONLY EVERYBODY IN THIS ROOM BUT WHOEVER IS LISTENING TO THINK ABOUT HOW COMPANIES ARE TAKING OUR DATA AND TRYING TO HELP SOLVE PUBLIC PROBLEMS OR ENABLE INSIGHTS THAT WE CAN'T DO HONESTLY, WE JUST DON'T HAVE THE SCOPE TO DO IT. SO YOU HAVE LOOKING, THE DATA FROM THE AMERICAN COMMUNITY SURVEY THAT POSTED IT IN WAYS THAT CAN BE ANALYZED EASILY ON A CENSUS, I THINK IT WAS CALLED -- ACSCS LOOKER, BUT YOU CAN SEE INSIGHTS IN THE DATA VERY QUICKLY EVEN ON THE FRONT END, THEY POSTED SPECIFIC QUESTIONS, ANSWERED THEM AND YOU CAN SEE THE DATA BEHIND IT. JUST THE IDEA THAT YOU CAN MAKE OUR DATA ACCESSIBLE VISUALIZABLE, THAT QUICKLY AND USABLE IS INSPIRATIONAL TO ME AND THEY DID IT FREE AND OPEN FOR THE PUBLIC. FS SOFT IS TAKING SOME OF OUR PTO IMAGES IN TIFTS AND APPLYING THEIR OCR AND MACHINE LEARNING TECHNOLOGY AND WILL START BY MAKING THE DATA SETS YOU CAN ANALYZE THE UNDERLYING DATA SETS FREE AND OPEN FOR THE PUBLIC WHICH OTHER COMPANIES ARE ALSO DOING BUT YOU CAN -- THEY'RE LOOKING AT THE NEXT STAGE, HOW WE CAN LOOK FOR INSIGHTS INTO THE DATA AND DRIVE USE IT. I WILL LET ANTHONY TALK ABOUT WHAT KAGEL IS DOING AND KEVIN TALK ABOUT SOCRATA HIS FIRST ONE, POSTING THE THEIR NEW PLATFORM AND HOSTING SOME OF THEIR DATA SETS. DATA.WORLD IS COMMITTED TO TAKING A NUMBER OF CENSUS DATA SETS INCLUDING ARCCS PUT THEM ON THE PLATFORM FOR FREE AND OPEN ACCESS COMBINE THEM AND MAKE THEM INTEROPERABLE ON AN API THAT PEOPLE CAN USE WITH OTHER OPPORTUNITIES PROJECT DATA SETS, THIS IS AGAIN SOMETHING WE'RE NOW LEADING THE OPPORTUNITY PROJECT, THE IDEA THAT YOU CAN COMBINE DATA SETS ON A FREE AND OPEN PORTAL IS TERRIFIC. THAT'S EXCITING. SO ANTHONY, I MIGHT TURN IT OVER TO YOU ON CAGLE IF WE CAN GET ANTHONY UP. ANTHONY, WELCOME. >> ANTHONY, WE CAN HEAR YOU, GO FOR IT. >> MAYBE JUST BEFORE I KICK OFF I'M CURIOUS, DO PEOPLE IN THE ROOM HAVE YOU HEARD OF CAGLE BEFORE OR IS THIS A NEW INTRODUCTION? COOL. EXCELLENT. I CAN'T HEAR BY THE WAY NOW. SO I'M ASSUMING YOU CAN HEAR ME. I'M JUST GOING THE TALK BUT IF I'M IGNORING ANY QUESTIONS BECAUSE I CAN'T HEAR. SO WHAT WE HAVE DONE WITH -- SO JUST FOR THOSE WHO AREN'T FAMILIAR WITH CAGLE WE'RE A COMMUNITY IN A FEW DAYS IT WILL BE OUT OF 700,000 DATA SCIENTISTS AND ORIGINALLY WHAT THEY DID, WHY THEY CAME TO CAGLE WAS TO COMPETE IN MACHINE LEARNING AND DATA SCIENCE COMPETITION. WE HAVE NOW ABOUT TWO MONTHS AGO OPENED UP AN OPEN DATA PLATFORM AND ACTUALLY THE U.S. CENSUS BUREAU IS THE FIRST GOVERNMENT AGENCY WE PARTNERED WITH TO START TO GET DATA SETS IN FRONT OF OUR COMMUNITY. SO WHAT THIS LOOKS LIKE, THIS IS U.S. CENSUS BUREAU PAGE, IT PIECE A BIT OF A DESCRIPTION OF THE CENSUS BUREAU AND DATA SETS BEING PUT UP. IF I GO INTO A DATA SET, A 2013 AMERICAN COMMUNITY SURVEY, NOT ONLY DOES IT HAVE A DESCRIPTION OF THE DATA SET BUT WE HAVE A TOOL THAT ALLOWS PEOPLE TO SHARE ON JUPITER NOTEBOOKS AND OTHER WAYS TO SHARE ANALYSIS PUBLICLY. SOMEBODY HAS WRITTEN ANALYSIS ON SHOULD I DO A Ph.D., SOMEBODY ELSE IS DONE ANALYSIS ON FROM THE AMERICAN COMMUNITY SURVEY AND IF I CLICK ON THOSE, SHOULD I DO A Ph.D., THEY HAVE WRITTEN UP THEIR WORK AND BASICALLY GOT CONCLUSIONS, HERE IS THE CODE, THEY SHOW CHARTS, SO RATHER THAN LAUNCHING A DATA SET, WE THINK DATA.GOV IS A GREAT VERSION ONE OF OPEN DATA AND THIS IS A VERSION 2 WHERE PEOPLE CAN SHARE CODE THEIR ANALYSIS ON TOP OF THE DATA AND SO IT CREATES MUCH MORE A VIBRANT ECOSYSTEM AROUND THE DATA. SEW SAY I WAS INTERESTED IN THIS SHOULD I DO A Ph.D. ANALYSIS? I CAN HIT FORK AND WHAT ALLOWS ME TO DO GET THIS PERSON'S CODE, RUNNING IN AN ENVIRONMENT WHERE I CAN RUN IT AND I CAN GO IN AND START EDITING THE CODE AND ONCE I'M FINISHED I HIT RUN AND I HAVE A IMPROVED VERSION OF THAT PERSON'S CODE. SO IT MEANS THAT SOMEBODY ISN'T JUST -- THE DATA ISN'T JUST BEING SENT OUT, WE DON'T SEE WHAT PEOPLE DO WITH IT BUT RATHER THE ANALYSIS IS PAIRED WITH THE DATA IN THE CODE IS SHARED IN A REPRODUCIBLE WAY SO PEOPLE CAN (INAUDIBLE) FROM EACH OTHER AND BUILD ON TOP OF WHAT EACH HAS DONE. YOU CAN SEE THIS PARTICULAR PIECE OF ANALYSIS IS FORKED 85 TIMES SO 85 PEOPLE HAVE TAKEN THE SHOULD I DO A Ph.D. ANALYSIS AND BUILT ON TOP OF IT, SAME GOES FOR ANALYSIS OF WORKING MOUNTAINS, WHETHER THEY ARE -- WHAT IS MARITAL STATUS, INCOME LEVELS. SO THIS WAS A PIECE OF ANALYSIS, WRITTEN BY A WORKING MOM THAT WAS CURIOUS ABOUT THE ANSWER TO THE QUESTION AND YOU CAN SEE IT'S QUITE A NICELY DONE ANALYSIS, LOTS OF CHARTS, LOTS OF VISUALIZATION, LOTS OF DESCRIPTIONS, GRAPHIC MAPS SHOWING THE ACTIVITY IN DIFFERENT PARTS OF THE COUNTRY AND LIKE PIECE OF ANALYSIS ENDS WITH A SUMMARY. WHAT WE DO, WE STARTED PUTTING UP SOME OF THE MOST INTERESTING DATA SETS FROM THE U.S. DEPARTMENT OF COMMERCE IN PARTNERSHIP WITH JUSTIN JEFF AND TERI. AND THE MOST INTERESTING DATA SETS THAT GET PUT UP ON CAGLE WE FEATURE ON A DATA SETS PAGE SO YOU CAN SEE HERE THE AMERICAN -- 2014 AMERICAN COMMUNITY SURVEYS CURRENTLY ON TOP, NO SURPRISE FROM THE U.S. PRESIDENTIAL ELECTION DATA IS UP, WE HAVE GOT BRAIN WAVE DATA, CRIME STATISTICS, SO THIS IS LIKE -- THIS PAGE IS A VIBRANT PLACE FOR PEOPLE LOOKING TO DO ANALYSIS AND COME AND FIND INTERESTING DATA SETS SO WE FEATURE THE MOST INTERESTING, WE FEATURE -- WE HAVE 126 FEATURE DATA SETS, MANY MORE WE DON'T FEATURE. ONCE PEOPLE DISCOVER SOME INTERESTING DATA SETS FROM THE U.S. CENSUS BUREAU, THEY CAN CLICK AROUND AND SEE OTHER DATA SETS. TO GIVE YOU A SENSE WE HAVE A ROUND ABOUT 125 TO 150,000 PEOPLE VISITING THIS PAGE UNIQUE PEOPLE VISITING THIS PAGE EVERY MONTH. THIS IS ONLY OUR WEAPON DATA PLATFORM IS OLD TWO MONTHS OLD SO THE BIG THING WE ARE DOING HERE IS NOT JUST LAUNCHING DATA SET DATA.GOV IS A GOOD START BUT WE ARE MOVING TO THE LAUNCHING DATA SET TO EMPTY ROOM TO A VIBRANT COMMUNITY WHERE PEOPLE CAN SHARE THEIR ANALYSIS, BUILD UP EACH OTHER ANALYSIS, I CAN SAY THAT WE PUSH THIS OUT AS JUSIN PULLED OUT OF THE MEETING AND HAD A REALLY GOOD RECEPTION. NOT SURE IF JOY IS IN THE ROOM BUT WE'RE TALKING TO SAN FRANCISCO ABOUT DOING SOMETHING SIMILAR WITH THEIR DATA SETS AS WELL AS THE WORLD ECONOMIC FORUM SO I THINK THIS IS BY PARTNERING WITH THE DEPARTMENT OF COMMERCE, WE MADE IT OKAY FOR OTHER GOVERNMENT ORGANIZATIONS TO DO THE SAME AND I THINK THAT NOT ONLY DO I INTERESTING ANALYSIS TO BE DONE WITH THE DEPARTMENT OF COMMERCE DATA BUT I THINK THIS PROJECT WILL SHOW THE WAY FOR OTHER GOVERNMENT ORGANIZATIONS THAT WANT TO RELEASE DATA INTO A VIBRANT COMMUNITY AND SEE WHAT'S GETTING DONE. I WILL STOP THERE. I'M NOT SURE -- I DON'T KNOW WHAT THE FORMAT IS, IF WE'RE TAKING QUESTIONS, NOT SURE I'LL BE ABLE TO HEAR BUT WE CAN TRY. SO THAT DOES POSE A CHALLENGE, ANTHONY. WE'LL HAVE A DISCUSSION, THAT WAS A TERRIFIC, WE'RE VERY EXCITED ABOUT THE IDEA COMMERCE DATA BEING HIGHLIGHTED IN THAT COMMUNITY. VERY THOUGHTFUL PRESENTATION, KEVIN I WANT TO TURN IT TO YOU BEFORE WE GET TO DISCUSSION. THERE'S SO MUCH TO TALK ABOUT. KEVIN, COULD YOU GIVE A LITTLE BIT ON WHAT SACRADA HAS DONE? >> JUST TO GIVE YOU A QUICK SIDE BY SIDE COMPARISON, THIS IS WHETHER THE DATA LOOKS LIKE WHEN YOU GO TO THE CENSUS BUREAU'S WEBSITE SO WE MAKE IT AVAILABLE BUT IT'S JUST YOU CLICK ON A BULK DOWN OR API FILE. >> THANKS, LAURA. SO WE TOOK THE SMALL AREA HEALTH INSURANCE ESTIMATE DATA FROM CENSUS WHICH LAURA SHOWED YOU IS AVAILABLE AS A DOWNLOAD ON CENSUS WEBSITE AND WE FED THIS INTO A PLATFORM THAT SACRATA HAS BEEN RUNNING THE LAST COUPLE OF YEARS AS OPEN DATA NETWORK. WE WORK WITH 1,000 OR 1100 GOVERNMENT CLIENTS AND A FEW NON-GOVERNMENT CLIENTS LIKE WORLD BANK YOU KNOW AND EUROPEAN COMMISSION MOSTLY AROUND THE FIRST AND THE LAST MILE OF GETTING DATA ONLINE FROM THESE ORGANIZATIONS. SO THE OPEN DATA NETWORK IS AN INITIATIVE FROM A COUPLE OF YEARS AGO AND IT HAS ONE PRIMARY OBJECTIVE AND TWO KINDS OF DATA IN IT. THE OBJECTIVE TO MAKE DATA UNDERSTANDABLE FROM THE LENS OF COMPARISON SO IF YOU COMPARE ONE COUNTY TO ANOTHER OR ONE STATE TO ANOTHER OR ONE KIND OF CRIME TO ANOTHER, OPEN DATA NETWORK DOES THAT QUITE NICELY. THE TWO SOURCES OF DATA IN THE OPEN DATA NETWORK ARE EITHER LOCAL DATA SETS, THAT ARE CONCATENATED BY THE NETWORK ON THE BACK END SO WE DO MACHINE LEARNING AND STANDARDIZATION AND NORMALIZATION TO GET THE DATA INTO A CONSISTENT FORM OR IN THE CASE OF THIS DATA SET, IT'S A NATIONAL DATA SET THAT HAS SOME REGIONAL CHARACTERISTICS TO IT, THAT WE ARE ABLE TO PRESENT IN THIS CONTEXT. SO LET'S DIVE IN AND SHOW WHAT WE'RE DOING. WE'RE TAKING THE DATA SET, I'M GOING TO GO BACK TO THE START SO THAT IT MAKES MORE SENSE. AND LET ME JUST COMPARE THE HEALTH INSURANCE RATES FOR TWO DIFFERENT STATES. SO I'M GOING TO ENTER MASSACHUSETTS AS A STARTING POINT, AND IT KNOWS IT AS A REGION, SO BY DEFAULT IT PULLS UP BASIC DEMOGRAPHICS. THIS IS ALSO COMING FROM THE DEPARTMENT OF CENSUS, BUT IN THIS CASE I WANT TO GO OVER AND I WANT TO LOOK AT HEALTH DATA, THIS IS WHERE THE HEALTH INSURANCE DATA IS SO I CAN SEE OVER THE LAST TEN YEARS OR SO, THE RATE OF PEOPLE UNINSURED IS DIMINISHING. THAT'S MASSACHUSETTS IN ISOLATION, SO HOW MIGHT THAT COMPARE TO ANOTHER STATE IN THE COUNTRY? LET'S GO DOWN TO GEORGIA AND I WILL COMPARE GEORGIA, IT SHOULD ZOOM THE MAP OUT A LITTLE BIT AND COMPARE MASSACHUSETTS UNINSURED RATE COMPARED TO GEORGIA. SO ALL THE VISUALIZATION, ALL THE TEXT, ALL THE GRAPHICS, ALL THE CHARTS ARE AUTOMATICALLY GENERATED. SO EVEN THESE PARAGRAPHS PAIR PERCENTILE WITHOUT HEALTH INSURANCE 22.2% FOR ALL RACES BOTH SEXES, THE PERCENT WITHOUT HEALTH INSURANCE IN MASSACHUSETTS WAS ONLY 4.5%. SO HERE, AGAIN, TRYING THE MAKE THE DATA USABLE, USEFUL NARRATIVE IS APPROPRIATE FOR AUDIENCES VISUALIZATIONS ARE APPROPRIATE FOR AUDIENCES, API ARE APPROPRIATE FOR AUDIENCES, AS YOU CAN SEALANT LEFT THE HARD TO SEE ON THE SCREEN BUT THIS GREEN BUTTON IS API BUTTON. IF YOU WANT PROMMATIC ACCESS TO THE DATA YOU CAN. SO SIMPLE WAY TO COMPARE DATA BETWEEN STATES, IF I WANT TO CHANGE DIMENSIONALITY, IF I WANT TO CHANGE THE DIFFERENT YEAR OR MAYBE NOT INTERESTED IN 18 TO 64-YEAR-OLDS, I'M INTERESTED IN 50 TO 64-YEAR-OLDS, I CAN CHOOSE THAT, IT WILL REGENERATE THE CHARTS IN THE MAPS. I CAN SEE THAT INFORMATION. INSTEAD OF STATE LEVEL LET'S DO COUNTY LEVEL. COB COUNTY HERE IN GEORGIA. THEN I'M GOING TO PICK ANOTHER COUNTY UP IN MASSACHUSETTS, LET'S PICK MIDDLE SEX COUNTY. SO NOW JUST LIKE I COULD AT THE STATE LEVEL, I CAN COMPARE AT THE LOWCAL LEVEL SO YOU CAN DO IT BY COUNTY OR METROPOLITAN STATISTICAL AREA. SO HERE IF I WANT TO LOOK INSTEAD OF A VISUAL HEAT MAP FORM OR NARRATIVE FORM, I CAN SEE -- LET ME GO TO HEALTH. I CAN SEE THE UNINSURED RATES AT THE LOCAL LEVEL ADS WELL. IT'S A TELLING STORY, BLUE LINE IS COB COUNTY DOWN IN GEORGIA WHERE CURRENTLY PERCENTAGE OF PEOPLE WHO DON'T HAVE INSURANCE IS ABOVE 20%, # 2 1/2%, MUCH LOWER RATE THAN WHAT WE SEE IN GEORGIA. GREAT WAY TO DO COMPARISON, YOU CAN COMPARE MULTIPLE COUNTIES MULTIPLE STATES WITH EACH OTHER, LAST THING I WOULD SHARE 7 OR 8 YEARS OF WORK THAT WENT INTO BUILDING THE GENERAL YOU SEE HERE TODAY BUT IN TERMS OF TAKING DATA SET MAKING USEFUL FOR THIS FORMAT, IT TOOK THREE OR FOUR HOURS, SO TAKING NEW DATA SET AND FEEDING INTO THIS COMPARISON ENGINE IS STRAIGHT FORWARD. >> TERRIFIC. MY POINT BEFORE I OPEN UP TO DISCUSSION IS YOU CAN SEE HOW EXCITED WE ARE. THIS IS AN IDEA WHERE WE TOOK THE DATA, WE'RE TRYING TO CURE RATE IT, MAKE IT PUBLIC, RAISE THE PROFILE OF IT. YOU SEE RELEASING THROUGH MEDIAN POSTS AND BLOGS AND TWEETS, HERE IS INTERESTING DATA SETS, HELP US GET IT OUT THERE, BUT YOU SEE COMPANIES STEPPING UP AND TRYING TO SOLVE THE PROBLEMS WITH USER GROUPS THAT THEY KNOW EXIST. THAT'S EXCITING. SO JOIN US. TAKE THE CHALLENGE, FINDS A SOLUTION OR PROBLEM YOU CARE ABOUT, TO TRY TO POINT YOU TO THE DATA SETS, THESE ARE OPEN PUBLIC DATA SETS, MY VIEW IS IT'S SORT OF LIKE US SAYING WE HAVE THIS ROAD, HERE IS THE ROAD. USE IT TO DELIVER FOOD TO THE POOR, USE PUBLIC ASSET FOR SOME PUBLIC OPEN INTEREST GOOD AND WE'D LOVE TO TRY TO TALK ABOUT AND SHOW OFF EXAMPLES LIKE SOCRATA, CAGLE AND THE OTHER COMPANIES, LOOKER AND SOME OF THESE COMPANIES ARE DOING TO TRY TO OPEN UP FOR THE PUBLIC GOOD. WITH THAT, I THINK WE HAVE THREE DISCUSSANTS ON THIS ONE, DANIEL, ALLEN AND COLIN. MAKE YOU CAN START. >> AMAZING STUFF, IT'S A TOUGH PROBLEM TO GET THIS STUFF OUT THERE. I WAS WANTING TO SHARE A QUICK STORY FROM MY EXPERIENCE AT LINKEDIN WHICH MIGHT BE INSTRUCTIVE. IN 2004 WE RELEASED THE COMPANY, NO ONE HAD ANY IDEA WHAT TO DO WITH IT. SO WE GO TELL PEOPLE, HEY, YOU CAN SOLVE ANY BUSINESS PROBLEM USING LINKEDIN AND YOU GO I DON'T BELIEVE YOU, SHOW ME. BIDS PROBLEM BUT PEOPLE TELL US THE BUSINESS PROBLEM TWO PEOPLE THE TOP OF THE LIST TWO FRIEND OF A FRIEND WHO WOULD BE ABLE TO HELP SOLVE THAT PROBLEM, PEOPLE GO WOW THAT'S AMAZING. THEY ARE NOT INTERESTED IN ACTUALLY DOING IT. THE LESSON WE TOOK FROM THAT IS PEOPLE AREN'T SIMPLY NOT CREATIVE, THEY NEED GUIDANCE. IN TERMS OF WHAT THEY'RE ACTUALLY GOING TO DO. SO YOU CAN'T GIVE SOMEBODY A COMPLEX TOOL AND EXPECT THEM TO COME AWAY WITH RESULTS THAT YOU SPECIFICALLY TRY TO DRIVE. SO WHAT THAT MEANS, THAT YOUR GOAL IS REALLY TO GET NON-PROFITS USING IT MORE, YOU HAVE TO FOCUS PEOPLE'S ATTENTION ON SOLVING PROBLEMS FOR NON-PROFITS. IF YOU DON'T YOU'RE GO FOG GET THE REAL ESTATE INTEREST WHO SEES IMMEDIATE -- NO OFFENSE, REAL ESTATE INDUSTRY WHO SEES IMMEDIATE VALUE, THEY WILL COME IN AND USE IT BUT PEOPLE WHO DID NON-PROFIT STUFF WON'T. IT ALSO MEANS THE REAL WAY YOU'RE GOING TO GET OUT IN FRONT OF A LOT OF PEOPLE IS YOU'RE NOT -- YOU CAN'T COUNT ON LOTS OF NON-PROFITS USING THE DATA DIRECTLY. AGAIN, THEY JUST DON'T HAVE THE TECHNICAL EXPERTISE SO IF YOUR GOAT GOAL IS 10,000 NON-PROFITS USING IT, I PICKED THAT NUMBER OUT OF THE AIR, THE ONLY WAY YOU GET 10,000 NON-PROFITS IS TO SOMEHOW ATRACK TWO OR THREE TEAMS TO BUILD TOOLS WHICH ARE GOING TO BE INCREDIBLY SELF-EVIDENT TO THOSE NON-PROFITS THEY NEED TO GO WOW, A GREAT FUNDRAISING TOOL I'M GOING TO USE. ONE OF THOSE TEAMS COULD BE SAY FROM SOMEBODY BUILDING ON TOP OF SALESES FOR, A LOT OF NON-PROFITS USE SALESES FOR AS MECHANISM FOR FUND RAIDSERS SO THEY'RE TURNING TO THAT ENVIRONMENT. MY RECOMMENDATION ON THIS STUFF WOULD BE UNDERSTAND THAT MOST PEOPLE DON'T HAVE THE -- AREN'T GOING TO APPLY CREATIVITY NECESSARY TO REALLY RING OUT THE API, YOU WON'T GET A 10TH OF A PERCENT OF THE POSSIBLE VALUE. SO THE BEST THING TO DO IS PICK AREAS YOU WANT TO FOCUS IN, THE PROBLEMS YOU THINK YOU SHOULD BE SOLVING WITH THE DATAING CHALLENGES AROUND THOSE THINGS TO ATTRACT TWO OR THREE PEOPLE WHO COME IN AND TRY TO SOLVE THE PROBLEM. IDEALLY, IT'S A PROBLEM NON-PROFITS IN THIS CASE OR LAWYERS PUBLIC DEFENDERS OR SMALL AND MEDIUM MANUFACTURERS OR WHATEVER GROUPS YOU CARE ABOUT, PUT IT IN FRONT AND FIND THAT OUT. THE ONLY CAVEAT I WOULD ADD IS MAKE SURE YOU'RE MEASURING THE OUTCOME, IF YOUR GOAL IS SERVING NON-PROFITS THAT'S GREAT BUT KNOW THE NUMBER OF API CALLERS IS THREE. NOT 10,000 SO MEASURE END USERS THAT REQUIRES YOU TO HAVE SOME WAY OF MEASURING WHO PEOPLE PROVIDING THE SOLUTION. >> COLIN. >> I'M IN A SIMILAR PLACE TO ALAN AND DEMOCRATIZATION. WHAT I COME FROM THE ONLY WAY WE GET AROUND IT WE FOUND IS WE HAD TO USE A MEDIUM EVERYONE ELSE USED SO WE HAD TO FIND WAY TO PUT IT IN AN APP THAT YOU CAN USE IN APPLE OF ANDROID SO SOMEBODY HAD TO GUILTY IT AND PUT IT ON THAT. SO -- GAMEF GAMEFY IT. EVERYONE IS INTERESTED IN ENERGY, EVERYONE IS INTERESTED IN THINGS LIKE IF I GO TO SOLAR ENERGY ON MY HOUSE, IS IT GOING TO BE USEFUL? SO IF THERE'S DATA AROUND HOW MANY PEOPLE WOULD DO THAT OR OTHER PEOPLE THAT ARE DOING IT, A PURPOSE, AND YOU HAVE AN APP UNYOU CAN DOWNLOAD AND FIGURE IT OUT, IT WILL HAPPEN. I LIKE THE NOTION OF FINDING ONE OR TWO THINGS VALUABLE TO PEOPLE AND WE CAN GAMEFY TO DO IT, SET A CHALLENGE AND SAY WE DO HACKATHN BASED ON THIS IS A BEAUTIFUL INTERFACE, LET'S HAVE A HACKATHON, THE TOP TEN PEOPLE GET SOMETHING TO -- WE'LL HELP YOU DO THAT. I'LL VOLUNTEER TO HELP YOU DO THAT. AND THEN BASED UPON THAT, WE TRY TO SEE IF WE CAN GET SOME APPS THAT PEOPLE CAN USE, PEOPLE, IT'S VERY HARD TO USE THESE TOOLS UNLESS YOU USE THESE TOOLS. BUT I THINK IT'S SOMETHING WORTHWHILE ENOUGH FOR US TO HAVE TO DO BECAUSE WE HAVE TO GET THAT EQUALITY GOING. SO THAT'S ONE SUGGESTION I WILL HAVE. BEAUTIFUL TOOLS, THAT SOLVES HALF THE PROBLEM RIGHT THERE. >> HECK OF A CONVERSATION. I THINK THIS -- ALSO -- CONVERSATION BEFORE WITH DANAH, I THINK THERE'S FOUR QUESTIONS, ONE IS DIRECT VERSUS INDIRECT IMPACT AND ULTIMATELY IT'S AT SOME POINT WE HAVE TO AS AN ORGANIZATION OR -- THEN BE FOCUSED ON THIS IS THE -- A LOT OF CHALLENGES YOU WILL ALWAYS -- IF YOU DON'T COMMIT TO SOMETHING YOU'LL COME BACK TO THESE HIGH LEVEL -- ALL THE TIME WHICH IS DO WE CARE ABOUT THIS, OR CARE ABOUT THIS, YOU SAY OKAY, THIS IS THE ONE WE CARE ABOUT, LET'S DO THAT HAS AS WELL AS WE CAN AND EXECUTE ON IT. AT LEAST SHORT TERM THAT WILL BE A CHALLENGE FOR TO PICK THE POSITION AND USE THAT TO DEFINE ITSELF. >> SO I THINK THAT POINT TOO, ON THIS ONE I THINK THIS IS PART BUT HOPING SMALLER PART WHAT CBS HAS TO DO AND LARGER PART PRIVATE SECTOR DOES. IT'S AN ASSET ALREADY THERE TO HIGHLIGHT ELEVATE AND GET THE TOOLS BUT BOTH POINTS, COLIN, ONE OF THE POINTS THAT'S INHERENT IN WHAT YOU WERE SAYING, THE CRITICAL ISSUES WHO ARE WE DEMOCRATIZING TO. ARE WE TRYING TO SAY EVERY TEAM BEING IN THE UNITED STATES -- HUMAN BEING IN THE UNITED STATES NEEDS TO ACCESS AND USE DATA? RIGHT NOW THAT'S NOT THE GOAL, I DON'T THINK IT'S A REALISTIC USE OF IT. WHAT WE'RE TRYING TO THINK ABOUT IS SOMEWHAT SOPHISTICATED USERS THAT MIGHT BE ABLE TO BRING INSIGHTS TO THE DATA THAT MIGHT HAVE LEVERAGE FOR OTHERS. SO POINTS YOU ARE MAKING ARE GOING TO THAT ISSUE. SO IN SOME WAYS WHEN WE SAY CAN WE GET SMALL MEDIUM SIZE ENTITIES TOOLS SCALABLE TOOLS THAT THEY CAN ACTUALLY BRING INCITES TO ACS DATA FOR EXAMPLE, WHAT I'M THINKING ABOUT THERE, WHAT WE HAVE BEEN THINKING ABOUT THERE, IS SOPHISTICATED XL USERS USERS, DON'T MEAN TO PICK A PARTICULAR COMPANY BUT SOPHISTICATED USERS IN A MEDIUM SIZE COMPANY THINKING ABOUT WHERE I'M MARKETING WHO ARE MY CLIENT BASE, THAT KIND OF THING. THAT'S WHAT BIG COMPANIES ARE DOING, THAT'S WHAT SOPHISTICATED USERS O OF THE DATA ARE DOING. WE NEED COMPANIES THAT KNOW HOW TO DO THAT WE HAVE DATA AVAILABLE SO WE HOPE TO GET ABATIC ON ISSUES LIKE THAT. FOR ME THE REAL OPPORTUNITY FOR ME OUT OF A CAGLE IS EXPERTS THAT GETS PROFILE OR ADVOCACY GROUPS, THEY MIGHT NOT BE ABLE TO SEE INSIGHTS ON HOW THEY OUGHT TO BE ADVOCATING THEIR OWN USERS THAT EXPERT DATA SCIENTISTS PART OF THE CAGLE COMMUNITY BRING INSIGHTS TO THEM AND THEY CAN USE THE INSIGHTS WITHOUT HAVING TO HAVE DONE THE WORK. FOR ME, SOMETHING LIKE A TOOL THAT SOCRATO BUILD, IF I HAVE POLICY MAKERS AROUND THE GOVERNMENT, NON-PROFIT OR OTHER SPACE, THE IDEA THAT WHEN THEY'RE -- INSURANCE IS A BIG ISSUE THE IDEA YOU CAN EASILY BRING INSIGHTS AND VISUALIZATIONS TO THAT, WE NEED TO MARKET THESE THINGS. WE NEED TO KNOW WHAT THE INSIGHT INSIGHTS YOU CAN BUILD ON SAFETY DATA AND THE IDEA IT'S ACCESSIBLE IS EXCITING TO ME BUT THERE'S CHALLENGE, I AGREE THERE'S A LOT TO DO. THERE'S ALSO INSPIRATION HERE. THERE'S REAL INSPIRATION THAT IF I WOULD HAVE TOLD YOU TWO YEARS AGO WE'RE SITTING IN AN OFFICE, IN A MEETING LIKE THIS AND YOU'RE SEEING TOOLS INTERNAL AND EXTERNAL THAT ARE BEING INSPIRED AND DRIVEN BY THE PRIVATE SECTOR AND INSPIRED AND DRIVEN BY OUR TEAM, WE WOULD BE GIVING EACH OTHER HIGH FIVE, THAT'S GREAT. WE HAVE MORE TO DO BUT THIS IS A GOOD START AND HOPE IT INSPIRES OTHERS TO STEP UP AND HYPOS. OTHERS WANT TO JUMP IN ON THIS BEFORE -- >> YOU MENTION OPEN SOURCE. I THINK YOU DIDN'T PAY ENOUGH ATTENTION TO IT. IN GENERAL TALK ABOUT MORE COMMUNITY PUBLIC OR PRIVATE COMPANY I THINK OPEN SOURCE IS REALLY GREAT TOOL GOING FORWARD. SOME QUESTION ON OPEN SOURCE EASY TO KICK OFF ON GET HUB OR WHATEVER, IT'S VERY HARD TO MAINTAIN OPEN SOURCE COMMUNITIES. MY RECOMMENDATION WOULD BE CONTACT PEOPLE, THERE'S A GOOD PART ABOUT OPEN SOURCE, THOSE PEOPLE WANT -- LOOK AT THE SAME INSPIRATION AND WANT TO HELP YOU, THERE'S A LOT OF PEOPLE WHO STARTED OPEN SOURCE AND PROJECT COMES INTO MIND. LOVE TO HELP YOU, LIKE TO GUIDE YOU SO I WOULD COMMIT TO (INDISCERNIBLE) BUDGET PROJECT AND SAY -- THE RIGHT WAY OR OPEN SOURCE COMMUNITY IN GENERAL, A FEW PEOPLE WHO CAN HELP US MAINTAIN OPEN SOURCE COMMUNITY. NOT MAINTAINING THE TOOL ON GET HUB ENOUGH OPEN SOURCE TOOLS BUT ACTUALLY THE PEOPLE WHETHER CAN MAINTAIN OPEN SOURCE COMMUNITY. SAME WITH CAGLE, IT'S GREAT DATA SETS ARE THERE, BUT YOU NEED TO MAINTAIN OPEN SOURCE OPEN DATA, WHATEVER SOME KIND OF OPEN COMMUNITY OF THE DATA SCIENTISTS BECAUSE YOU WILL HAVE PEOPLE SIMILAR WITH CAGLE THE DATA SCIENTISTS WANT TO DO IT FOR TWO REASONS ONE, IT'S A SOURCE -- THEY WANT TO BE SOCIALLY GOOD AND HELP WITH THE WORK THANT TO HELP OR SHOW ALL THE SKILLS TO FUTURE EMPLOYEE, TO THE FUTURE -- TO FUTURE -- HOW GREAT THEY ARE. SAME WITH OPEN SOURCE. WE HAVE THE SAME REASONING WHY PEOPLE WOULD WANT TO DRIVE BUT KEEPING THE COMMUNITY IS A LOT OF WORK. PRIVATE COMPANY HIRE PEOPLE, HIRE THE TEAM, INNER SOURCE WHICH IS VERY SIMILAR AND A LOT OF EFFORTS SPENT ON IT -- REALLY SMALL ONE. WE'RE TALKING ABOUT MEASURES AND IT WAS MENTIONED, IT'S GREATER TO SHOW HERE NUMBER OF -- AND YOU POINT OUT THAT 12% ARE MADE BY THE (INAUDIBLE) IMMUNITY. THEN YOU START TALKING NUMBER OR NON-PROFITS. NONE OF THOSE NUMBERS, DOES NOT TELL YOU HOW MANY NON-PROFITS WE'RE USING BECAUSE IT'S POSSIBLE THAT THE COMMUNITY MAKING API EACH AND EVERY NON-PROFIT IS MAKING TWICE A YEAR WHEN WE WANT TO DO FUNDRAISING, WITH A DIFFERENT NUMBER OF CALLS SO WE CAN (INDISCERNIBLE) NON-PROFITS ARE USING IT BECAUSE IT'S POSSIBLE THAT REAL ESTATE COMMUNITY MAKING EACH AND EVERY NUMBER VERSUS NON-PROFIT MAKING CALL TWICE A YEAR, WHEN THEY WANT TO DO FUND RAISING. VERY DIFFERENT NUMBER OF CALLS SO WE CAN -- DOESN'T TELL HOW MANY ANGIOPLASTS. SO HAVING COLIN WAS HAVING THE RIGHT MEASURE IS GREAT BUT ALSO MAKE SURE YOU'RE MEASURING THE RIGHT THING. >> FIRST POINT WAS THAT IN DIRECT USAGE VERSUS DIRECT USAGE IF WE'RE HALF WHATEVER API CALLS THE CENSUS DATA, LET'S BE CLEAR, THE COMMERCIAL SECTOR HAS NEVER HAD A PROBLEM ACCESSING REAL ESTATE DATA OR CENSUS DATA. THEY CAN -- THEY HAVE RESOURCE TO GET THAT. PEOPLE WHO PRIMARILY USE OUR DATA, ARE CONSUMER WHOSE DIDN'T HAVE ANY ACCESS BEFORE OR THE DERIVATIVE PRODUCTS ONCE WE PUT A CENSUS DATA POINT INTO A DATA NEGATIVE EQUITY PEOPLE WHO CONSUME THAT ARE PRIMARY CONSUMERS RESEARCHERS ACADEMICS, GOVERNMENT, GOVERNMENT INDUSTRY, GOVERNMENT SECTOR. FOR THE MEDIA. SO IT'S NOT COMMERCIAL USERS. IT'S THOSE COMMUNITIES. AND THOSE WOULD BE NON-COMMERCIAL USES PRIMARILY. SO YOU WOULD THINK, PROPERLY THOUGHT OF YOU COULD -- IF YOU SAW A DOT C OBJECTIONM CONSUMING WEB KEY REGISTEREDDED TO A DOT C O M CONSUMING THE DATA IT PROBABLY IS. THE OTHER THING I WOULD SAY PIGGY BACKING ON ALAN'S POINT ACCESSING, I DO THINK GENERALLY IT'S HARD TO PRESUPPOSE APPLICATIONS PEOPLE WILL USE THE DATA FOR AND BUILD FOR THOSE BUT I DO THINK DEMONSTRATING HOW IT'S USED IS USEFUL AND I WOULD JUST SAY FROM OUR PERSPECTIVE, WE HAVE A LOT OF API THAT EITHER SERVICE OUR OWN DATA OR THAT SERVE AS DATA THAT USES COMMERCE DATA BLENDED IN WITH IT AND THINGS THAT HAVE BEEN EFFECTIVE FOR US IN ORDER TO SPREAD THAT WORD OF THINGS LIKE ALMOST EVERY ORGANIZATION NON-PROFIT DO IT, JOURNALISTS DO, THEY HAVE A RICH COMMUNITY OF DATA JOURNALISTS THAT GET TOGETHER SEVERAL TIMES A YEAR, YOU CAN GET IN FRONT OF GROUPS AND SAY HERE IS SOME PYTHON CODE THAT DATA JOURNALISTS ARE A BUNCH OF SCIENTISTS THAT WORK FOR LARGE OR SMALL JOURNALISTS COMMUNITIES AND YOU CAN SHOW THEM HEY, HERE ARE THREE EXAMPLES HOW YOU PULL DATA IN TO TELL A STORY ABOUT NEGATIVE EQUITY OR HOME PRICES OR ANYTHING ELSE CENSUS RELATED. THAT'S EFFECTIVE. BECAUSE THEN THEY CAN WE WERE TALKING TO PEOPLE GIVING A FEW EXAMPLES OF THINGS THAT WERE DONE AND THEY CAN GO FROM THERE. >> WE'LL MOVE TO THE NEXT SESSION. WE'RE NOW RUNNING LATE. SO WHY DON'T WE START FUTURE OF CDAC DISCUSSION. DAN YOU WILL FACILITATE WITH CJ THAT DISCUSSION. WE'LL HAVE TO STOP FOR OPEN COMMENT OR PUBLIC COMMENT AT 4 O'CLOCK. IF WE HAVE COMMENTS WE'LL ENTERTAIN THEM, IF NOT WE'LL GO BACK INTO THAT DISCUSSION. >> LET ME PAUSE BY SAYING THANK YOU, THOSE WERE ALL ALONG THE BOARD REALLY THOUGHTFUL COMMENTS AND WE -- THIS IS ONE IN WHICH HONESTLY CDAC MEMBERS YOU CAN HELP. THERE'S ACTIONABLE STUFF HERE TO AMPLIFY OR THINK DIFFERENTLY ABOUT AND WE HAVE TO FIGURE HOW WE ACT ON SOME OF THE ADVICE WE GOT HERE TODAY. THANK YOU VERY MUCH. VERY THOUGHTFUL. >> SO THE POINT OF THIS LAST DISCUSSION ITEM, IS REALLY TO OPEN UP A CONVERSATION ABOUT FUTURE OF CDAC THERE'S TWO QUESTIONS HERE, WHICH COMPOSITION CDAC LOOK LIKE AND ANY CONCRETE ADVICE WHAT WE SHOULD BE ASKING. SO ON THE LATTER QUESTION, SOME OF THIS IS FEEDBACK ON THE AGENDA, SHE DID PROVIDE QUITE A BIT OF FEEDBACK ON THE AGENDA ITSELF AND THAT'S BEEN HELPFUL ALONG THE WAY. WE TRIED TO SHIFT THE AGENDA TO HAVE LIGHTNING TALKS TO HAVE MORE DIRECT INTERACTION, JUSTIN DID A LOT OF LET WORK AND TONYA EARLY ON TO MAKE SURE WE CAN HAVE THESE CONVERSATIONS AHEAD OF TIME AND BRIEFINGS, SO FIRST FEED BECOME ON THAT JUICEFUL TO PLAN THE NEXT MEETING ON WHAT YOU LIKE ABOUT IT, AND WHAT YOU DIDN'T, WHAT COULD BE BETTER. BUT THEN ALSO THE LAST COVERS WE'RE HAVING WHERE WE SHOULD HAVE PARTNERSHIPS, WE SHOULD BE LOOKING TO INVITE NEW CDAC MEMBERS PARTICULARLY WITH COMPANIES WHERE IT MAKES STAFF PARTNERSHIPS INVOLVED IN THESE CONVERSATIONS, WE'RE JUST IN GENERAL, ARE THERE CERTAIN SKILLS, IT WOULD BE GOOD TO HAVE A CDAC TO HAVE, WE DON'T HAVE MORE OF AND MAYBE THERE'S THOUGHTS THERE. SO THAT'S THE POINT OF THE DISCUSSION. I'LL STOP SEE IF ANYBODY WANTS TO ADD ANYTHING. I ALSO INVITE STAN TO WEIGH IN ON THIS. >> ONCE THING THAT'S A GOOD SEGUE, THERE'S THREE QUESTIONS THERE, THE THIRD IS ONE THAT MATTERS BECAUSE IT INFORM IT IS OTHER TWO BEFORE THAT, AND THAT IS THAT, WHAT SHOULD THE CDAC DO BE ASKING -- SHOULD WE BE ASKING THE CDAC TO DO FOR US. I THINK YOU ARE ASKING, I HAVE A LIST OF THINGS TODAY ALONE AND IT GOES TO SOME OF THE WORK DANIEL DID EARLIER OF THINGS YOU ARE ASKING. WHEN LOOKING AT THOSE THINGS, ONE THING I NOTICE IS THERE'S HIGH LEVEL ADVISERY COUNCIL TYPE OF THINGS AND VERY HANDS ON ACTIVE DIVE DEEP TYPES OF THINGS IF GIVEN THE CHANCE WE TALKED EARLIER ABOUT SECURITY AND ONE OF THE THINGS THAT'S MY BACKGROUND AND MO TYPE AND WHEN WE WERE TALKING ABOUT THAT, ONE THING THAT STRUCK ME, WE'RE NOT TALKING ABOUT NECESSARILY SECURITY BUT ONE PART OF THE PILLARS, THERE'S THREE PILLARS TO SECURITY, AND ONE PART WE'RE TALKING ABOUT IS INTEGRITY. THERE ARE SOLUTIONS FOR INTEGRITY OF DATA THAT COULD BE USED IN THIS CASE, IT'S IS NOT SOMETHING YOU ADVISE ON, YOU GO AND LOOK AT THE USE CASES AND DO ACTUAL WORK ON THAT. SO IN DECIDING THE FUTURE OF CDAC OR TYPES OF MEMBERS WHETHER IN CDAC OR SOME OTHER PART OF THE GROUP, WE MAY LOOK AT TRYING TO LOOK AT -- BE ABLE TO DIVE A LITTLE DEEPER GET OUR HANDS DIRTY, ROLL UP OUR SLEEVES, THIS CARRIES ON TO OTHER THINGS WE TALKED ABOUT, MARKETING, SHARING SOCIAL NETWORKS. THAT'S A HIGH LEVEL THING, IT'S EASY, IT'S EASIER FOR US THAN ROLLING UP OUR SLEEVES. I THINK HIRE ENGINEERING STAFF WAS ONE OF THE ASKS AND FROM THAT STANDPOINT IT'S HIGH AND LOW LEVEL, ONE OF THOSE THINGS WE CAN ASSIST TO FROM THE STANDPOINT OF TRYING TO SEND THE RIGHT PEOPLE THIS WAY, THERE'S THE OTHER SIDES WHERE MAYBE A LITTLE BIT MORE DIVING DEEP THERE ARE RESERVE PROGRAMMINGS, IT'S A STRETCH BUT MANY COMPANIES HAVE -- IF YOU GO ON RESERVE DUTY FOR THE MILITARY THEY CONTINUE TO PAY YOU AND YOU WORK FOR THE MILITARY. LEAVES OF ABSENCES IS AN THINGS LIKE THAT ARE POSSIBILITIES IN MANY COMPANIES. THAT'S SOMETHING WE SHOULD OPEN UP OR LOOK AT, POSSIBILITIES OF MEANS BY WHICH TO GET THE ROLE UP YOUR SLEEVES DO MORE THAN WHAT YOU CAN DO IN A DAY OR TWO ONCE ORDERED. WHETHER US OR OTHER MEMBERS THAT WE BRING ON. THESE ARE THINGS I LOOKED AT, METRICS IS ANOTHER ONE, DIVE DEEP INTO METRICS. RIGHT NOW SOME OF THE CHALLENGES WE HAVE IN BEING ABLE TO DEAL WITH METRICS IS YOU LOSE VISIBILITY AT CERTAIN POINTS MOSTLY PRIVACY ISSUE FROM A DATA PROVIDER LIKE AWS. WE IF A COMPANY IS STORING DATA ON A PLATFORM, WE'RE NOT GOING TO LOOK AT THE DATA. AND THEREFORE I I DON'T IF YOU MADE CHANGES AND PASSED ON WHAT THE ECOSYSTEM IS DOING WITH IT. THAT DOESN'T MEAN IF YOU HAVE A PIECE OF -- SOME DATA YOU'RE PLACING ON A PLATFORM THAT'S PART OF THE FREE DATA SETS PROGRAM, AND ONE OF YOUR TERMS OF SERVICE TO USE THAT DATA IS IT TO THEN GET FEEDBACK FROM THAT USER. THIS IS SOMETHING I THINK FROM A HIGH LEVEL WE COULD WORK TO COME UP WITH THOSE AGREEMENTS PUT THOSE IN PLACE, SO IT'S A REQUIREMENT OF USING THE DATA THAT CAN PROVIDE THIS INFORMATION. SO THERE IS SOME THINGS THAT CAN BE DONE IN THAT SPACE. THOSE ARE THINGS THAT I WAS TRACKING THROUGHOUT THE DAY. >> LESS TIME TO THINK ABOUT OVERALL THOUGHTS HERE BUT I GUESS SOME THINGS THAT DO OCCUR TO ME ARE THINKING ABOUT CDAC ITSELF IN THE FUTURE, IS -- I GUESS SEEMS LIKE CDAC IS PRIMARILY BEEN USED TO IN THREE DIFFERENT WAYS. ONE IS TO HELP WITH MARKETING ITSELF OF WHAT THE -- WHAT'S AVAILABLE FROM COMMERCE DIFFERENT DATA SETS AND HELP GETTING WORD OUT ABOUT THAT. RAISING VISIBILITY IN AN AREA THAT NEEDS NOR VISIBILITY GIVEN TO IT. OTHER IS HELPING TO INFORM, BRING OUTSIDE THOUGHTS ABOUT WHAT PEOPLE SHOULD BE DOING IN THIS SPACE AND PARTICULARLY PERHAPS THERE HAS BEEN A LOT OF FOCUS ON CDS ITSELF. AND HOW WHAT ROLE IT COULD SERVE AND THE THIRD COMPONENT HAS BEEN OUT OF THE THINGS THAT COMMERCE AT LARGE CDS NEEDS HELP ON, ARE THERE WAYS WE CAN HELP PROVIDE ASSISTANCE OR INSIGHT INTO THOSE PARTICULAR AREAS. I THINK THAT SEEMS REALLY THOSE THREE FUNCTIONS SEEM LIKE THEY'RE REALLY IMPORTANT AND THEY SEEM LIKE THEY SHOULD BE GOING FORWARD AS WELL. I DON'T THINK -- NOT SURE THERE'S DIFFERENT ALLOCATION ACROSS THOSE THREE AREAS, BOS ARE IMPORTANT. I THINK MAYBE -- BOTH ARE IMPORTANT. I THINK MAYBE IN THE EXTENSION OF CDAC IT MIGHT BE USEFUL TO THINK ABOUT IT MORE ALONG THE THREE LINES AND THINK ABOUT IT MORE LIKE ALMOST YOU WOULD THINK CORPORATE GOVERNANCE WHERE THE BOARD HAS A SIMILAR BOARD OF DIRECTORS FOR CORPORATION HAS A SIMILAR FUNCTION WHERE ITS JOB IS TO BRING STUFF INTO MANAGEMENT THAT MIGHT NOT BE THINKING ABOUT AND REVIEW STRATEGIC PRIORITIES AND SEE WHETHER GIVE INSIGHT AN FEEDBACK ON THAT. ALSO PURPOSE OF THE BOARD TO ASINCE THE COMPANY, THE MANAGEMENT OF THE COMPANY WHEN PROBLEMS COME UP HOW YOU RESOLVE THE PROBLEMS AND CAN THE BOARD USE THEIR NETWORKS TO HELP RESOLVE THOSE PROBLEMS. LOOKING AT JEFF AND JEFF, THE PRIORITIES, I CAN IMAGINE A FIRMER ATTACHMENT BETWEEN WHAT'S HAPPENING INTERNALLY TO ALAN'S POINT OR QUESTION ABOUT HOW DO YOU DETERMINE THESE PRIORITIES, IT WOULD BE PERHAPS INTERESTING TO HAVE SEEN FULL LIST OF THINGS YOU COULD BE DOING AND FOR US TO PROVIDE ASSISTANCE FOR THINGS THAT MAYBE THIS IS AN UNDER INVESTED AREA AND BE MORE HELPFUL TO YOU THOUGH AGAIN IF YOU DON'T NEED THAT HELP, I GUESS I DO THINK THIS IS PROBABLY A QUESTION FOR JUSTIN MOORE, I DO THINK IT FEELS LIKE IS THERE IS A DECENT FLOW OF INFORMATION FROM AREAS YOU NEED OUR HELP. I FEEL LIKE THAT'S AN OPEN AREA WHERE JUSTIN IS VERY TRANSPARENT ABOUT WE CAN REALLY USE YOUR HELP IN THESE AREAS, THAT'S REALLY HELPFUL. SO I WILL -- I ALREADY MENTIONED I THINK DURING THE -- DURING YOUR THE 2017 PRESENTATION, JEFF, KIND OF A CUFFING OF AREAS IT DOES FEEL LIKE WE COULD -- THERE'S A BIT MORE META STUFF CDS OR CDAC OR CONGRESS OUTS CAN HELP WITH, THOSE AREAS AROUND DISCOVERY AND SEARCH PROBLEMS, BY SEARCH BEYOND EXTENSIVE SEARCH NOT SEARCH ENGINE BUT THE PROBLEM WE HAVE GOT TO MULTIPLEX PROBLEM YOU HAVE A BUNCH OF PROVIDERS PROVIDE VIEWING DATA AND PEOPLE LOOKING FOR DATA. AND THERE'S NOT YET BECOME A MARKETPLACE FOR THOSE PEOPLE CAN FIND EACH OTHER EASILY. AND SOLVING THAT WOULD BE REALLY USEFUL AND I DO THINK THAT THAT -- IF -- AND THERE HAVE BEEN ATTEMPTS AT THAT BUT IF WE CAN SOLVE THAT, THAT'S FANTASTIC AND THAT'S A REALLY WEIGHTY PROBLEM SOLVED. THE OTHER ONE IS KIND OF HOW TO HARNESS LONG TAIL COMMUNITIES. I WAS INTERESTED IN ANTHONY'S PRESENTATION ABOUT HIS -- WHAT THEY'RE DOING THAT DOES INTRODUCE THE PROSPECT OF PROVIDING A PLACE WHERE LONG TAIL COMMUNITIES AND WHETHER KEVIN IS DOING AS WELL WITH OP DATA NETWORK. DOES THAT CREATE A CAMPFIRE PEOPLE CAN GATHER FOR PEOPLE WHO USE FISHERIES API. THEY GET COMMUNITY INFORM AND SAY HEY I LOVE THE EXTRA DATA POINT, CAN YOU EXPOSE THIS THAT WAY AND THAT'S THE KIND OF THING WHERE I FEEL LIKE COMMERCE HAS DONE A GOOD JOB ON LARGE USER GROUPS FOR CENSUS DATA PRODUCT OVERALL BUT PROBABLY A LONG TAIL THEY CAN'T CAPTURE EFFECTIVELY AND I WOULD LIKE TO TRY TO GET MORE TRACTION THERE. >> TO ONTO ONE OF THE -- TO ADD TO ONE OF THE POINTS, I'M GOING THE PAUSE SO BURTON CAN CALL FOR THE PUBLIC O -- Q. WE HAVE NO FORMAL SIGNUPS, I WILL ASK IF ANYONE IN THE ROOM WANTS TO SAY ANYTHING. I'M NOT SEEING ANYONE. I THINK ON TWITTER WE HAVEN'T SEEN A LOT OF OR ANY SPECIFIC COMMENTS OR QUESTIONS. SO WE'RE GOING TO JUST SAY THAT WE'RE OPEN FOR THE NEXT 15 MINUTES IF SOMEBODY WANTS TO COMMENT, STAND UP AND LET US KNOW ONE WAY OR ANOTHER. BUT I THINK WE CAN AHEAD WITH DISCUSSION. >> I WANT TO REACT TO ONE POINT, TO YOUR QUESTION, STAN, I WOULD TELL YOU I THINK ALL OF US FEEL WHEN WE HAVE ASKED FOR IN REACTION WE'RE GETTING HELP IN REACTIONS, THE FACT THAT DAN AND KEN WERE ABLE TO GET YOU THAT HAVE VERY BUSY SCHEDULED TO LISTEN TO THESE THREE BRIEFS WHICH I HOPE ARE HELPFUL, TRYING TO GIVE DEEPER DIVES AND MORE INFORMATION BECAUSE YOU CAN TELL THE SCHEDULE IS PACKED. AND BY THE WAY, I DID NONE OF THE WRANGLING. MY GOAL WAS TO HAVE THAT DONE AND IT WAS REALLY BURTON AND TONJA THAT DROVE THAT PROCESS BUT WE FEEL SUPPORTIVE WHEN WE NEED IT. REALLY. I WOULD TELL YOU BECAUSE WHILE WE HAVE TALENT AND FOLKS FOCUSED ON THE MARKETING PIECE, LITERALLY, YOU ALL PROBABLY KNOW PEOPLE THAT HAVE A MILLION OR 2 MILLION OR 20 MILLION LINKEDIN FOLLOWERS. I BELIEVE AT LEAST ONE MEMBER OF THIS TABLE IS ABLE TO DO THAT. AND WE HAVE OTHER FOLKS THAT KNOW PEOPLE THAT HAVE 20 MILLION TWITTER FOLLOWERS. AND OTHERS, RIGHT? I NEED TO LEVERAGE THOSE NETWORKS. I'M BEING QUITE TRANSPARENT WHICH IS I HAVE 5,000 AND THE SECRETARY AND OTHERS HAVE QUITE A BIT MORE THAN I DO. BUT 20 MILLION -- WE NEED VISIBILITY ON THAT LEVEL AND I NEED ADVICE FROM PEOPLE THAT KNOW HOW TO LEVERAGE THE NETWORKS. SO WE'RE TRYING TO BE OPERATIONAL ABOUT IT, IT DOESN'T NEED TO BE FOR EVERYTHING BUT WHEN WE FOR EXAMPLE ARE BRANDING A PARTICULAR PART OF WHAT WE'RE DOING AND TRYING TO ELEVATE DATA SETS, FOR EXAMPLE THAT WE'RE DOING, OR THE HIRING EFFORTS, YOU ALL HAVE THE ABILITY TO PUMP IT OUT TO NETWORKS AND IT MATTERS. IT CRAZILY MATTERS. WE GET REACTIONS EVERY TIME ONE OF YOU GUYS ACTUALLY LIKE OR FORWARD IT. BELIEVE IT OR NOT, IT MATTERS. SO WE'RE OPERATIONAL ABOUT IT. I WOULD TELL YOU THAT WOULD BE A PLACE IF YOU GUY CONSIST HELP US GET TO THE PEOPLE THAT HAVE ALL THE THINGS AND THEY JUST LOOK AT IT, THEY DON'T HAVE TO LIKE IT IF THEY DON'T LIKE IT BUT MOST OF THE PEOPLE THAT WE SHOW WHAT WE'RE DOING ARE INTERESTED IN IT AND THINK IT'S AN INTERESTING APPROACH. BUT THEY JUST DON'T NOTICE IT RIGHT NOW SO THAT'S A PARTICULAR VERY OPERATIONAL THING THAT WE CAN USE HELP ON. ON THAT FRONT. >> WONDER IF YOU CAN GO OUT OFRD ORDER. PHIL, MAYBE YOU AND ALSO I WANT TO ASK JEFF, SINCE YOU'RE IN SOME WAYS THE TRANSITION GUY, I THINK ABOUT THE NEXT MEETING YOU'RE GOING TO BE ONE OF THE KEY CONTINUITY POINTS HERE, MAYBE YOU CAN GIVE ALSO A REACTION WHAT WOULD BE USEFUL FROM YOUR PERSPECTIVE, WHAT YOU WOULD LIKE TO SEE, AND THEN I WOULD LIKE TO ASK THE SAME QUESTION OUTGOING ADVICE BUT LIKE YOU TO GO FIRST. >> WE TALKED ON A NUMBER OF OCCASIONS ABOUT THE CONCEPT OF AN ECOSYSTEM AND I NOTICE IT DIDN'T END UP IN THE MEMORANDUM FOR SECRETARY PARTICIPANT SERVE'S ONE KEY ITEM YET IT FLOWS MORE THROUGH THE DISCUSSIONS AND STAN JUST COMMENTED I TOOK YOUR COMMENTS AS BUILDING AN ECOSYSTEM, ALONG THAT ECOSYSTEM. I'M WONDERING IF WE NEED TO SOMEHOW RAISE THE PROVINCE OR CODIFY THAT IN SOME BETTER WAY THAN JUST LETTING IT FLOW DIFFUSELY THROUGH DISCUSSIONS. >> I'M GOING TO ADD ALAN I TOOK NOTES HERE, SO ALAN MADE A COMMENT EARLIER, HE GAVE THREE PERFECT EXAMPLES OF THINGS YOU CAN DO TO ACTUALLY IMPLEMENT THAT. NOT JUST KIND OF A BROAD BRUSH, I CAN GO BACK AND FIND, -- FIND THEM. >> I THINK I WROTE THEM DOWN. >> HE HAD GREAT SPECIFIC -- >> IN TERMS OF MARKETING. >> I THINK FROM CDAC, I DO AGREE THERE'S A META LAYER THAT'S ABSOLUTELY NECESSARY, AS I'M MOVING AWAY FROM DAY TO DAY TECHNICAL MANAGEMENT TO MORE HOW MODULES FIT TOGETHER AND -- I DO BELIEVE IT WOULD BE GOOD TO HAVE MORE GOVERNMENT ADVICE ON ALL MATTERS OF GOVERNMENT DATA GOVERNANCE ISSUES TO PROJECT GOVERNANCE, HOW TO, WHAT ARE WAYS THINKING ABOUT HOW PROJECT WILL MOVE LONG TERM AND FITS TO GREATER SCHEME OF THINGS. GUT CHECKING WOULD BE GREAT. SO I WOULD FOLLOW-UP WITH MORE OF YOU TO GET ADVICE LIKE COTY WITH VISUALIZATION ENGINEERING ISSUES, COLIN AND MACHINE LEARNING WITH WHETHER OR NOT CERTAINLY AS WE PARTNER WITH SO THERE'S A LOT OF OPPORTUNITIES TO CHECK IT OUT. I WILL MAKE IT A PERSONAL NOTE AND PUT IT INTO MY CALENDAR TO MAKE SURE TO CHECK IN. MORE REGULARLY. WE MOVE ALONG TOWARDS SOMETHING THAT IS REALLY AN ONGOING PARTNERSHIP. >> IT'S A WAY ELEPHANT IN THE ROOM NOT DISCUSSED MORE READILY IN MY OPINION. I HAVE TREATED THE CDAC IN MY PERSONAL LINE THAT'S MY BACKGROUND SO I HAVE HAD THAT INSTINCT FINGERPRINT ON THE AGENDA THE PAST FEW SESSIONS, IF THIS WAS A REAL BOARD MEETING, YOU'RE GOING TO HAVE A NEW CEO IN A FEW MONTHS, PROBABLY DOMINATE THE ENTIRE DISCUSSION, THAT'S NOT ME OR JUSTIN OR AUSTIN WHO WILL NOT BE AT THIS TABLE NEXT TIME. JEFF BRAD AND BURTON WILL. BUT ALSO NEW SECRETARY OF COMMERCE. WHICH IS ILLEGAL IN SOME LEVEL THE DIRECT ADVISOR TO THIS COUNCIL. SO I WOULD SAY THAT GIVEN THAT'S ONE OF THE CHALLENGING RISKS THAT COME UP WITH THE COMPANY PRESENTING TO MY BOARD, I THINK THAT IS ALSO -- GREATEST OPPORTUNITY FOR HOW THE CDAC CAN HAVE IMPACT ON DATA GOING FORWARD IS SET UP INSTITUTIONAL STRUCTURES TO CONTINUE ON CARRY ON BUT IT'S HOW WELL ARE YOU GOING TO CONNECT THE NEXT BATCH OF FOLKS THAT TAKE THESE SEATS AND IT IS A I CAN CANNEN AND EGG CONVERSATION -- CHICKEN AND EGG CONVERSATION. HYPOTHETICALLY THE NEXT SECRETARY OF COMMERCE COULDN'T CARE ABOUT DATA, ABOUT SEWERED AND WE HAVE DONE THINGS -- ABSURD BUT WE HAVE DONE THINGS IN THE HYPOTHETICAL TO SEE IT'S APPROPRIATE SO BUT IT'S POSSIBLE SO THAT IMPACTS OUR MISSION. IT'S POSSIBLE TO DEFINE WHAT THE PRIORITIES WOULD BE FOR ADVICE OPERATIONAL STRATEGY, SO FORTH. AND AGAIN, I DON'T EXPECT A MASSIVE DISRUPTION. WE HAVE ACTING ROLES AND FOLKS WITHIN THE BUREAU AND INSTITUTIONS ARE REALLY I THINK REALLY WELL CEMENTED. BUT TO SOME EXTENT LEADERSHIP MATTERS. SOME LEVEL WHOEVER IS AT THE HELM CAN DRIVE PRIORITIZATION, THAT'S WHAT WE WANT IN THE DEMOCRATIC TRANSITION OF POLITICAL LEADERSHIP. THAT'S THE STRUCTURE. SO THE REFLECTION I WOULD SHARE GIVEN THIS IS PROBABLY THE LAST TIME I WILL BE AT A TABLE LIKE THIS, IS ONE OF THANK YOU. ONE OF IMMENSE GRATITUDE, OF DEEP AND SINCERE RECOGNITION OF YOUR SUPPORT AND HOW WE WOULDN'T BE HERE WITHOUT Y'ALL. AROUND ALSO A REAL BEG AND CHALLENGE THAT YOU CAN CONTINUE YOUR FANTASTIC SERVICE WITH WHOEVER IS HERE NEXT. I WILL ALSO JUST WITH A LITTLE SIDE POINT NOT QUITE DONE YET. AND AS PENALTY PRITZKER TALKS ABOUT, A TRY@LON ATHLETE, FOR THE TIME THAT REMAINS FOR ME, ONE THING THAT WILL BE FOCUSED ON MY OWN FEW MONTHS IS RECRUIT MORE FOLKS IN THE DATA SERVICE NTIS, CAREER STAFF, BUILD UP FOR FOLKS LIKE JEFF AND BRETT AND ALL THE INNOVATORS HERE CAN CONTINUE CARRYING THROUGH THIS GREAT ACTIVITY REGARDLESS OF PARTICIPANTCAL TRANSITION IS IGNORING IT. THAT BEING MY FOCUS, I WILL BRIEF CALL, IF YOU KNOW ANY FOLKS THAT WANT TO DO A TOUR OF DUTY, WE TALK ABOUT FELLOWSHIPS, I WILL BE FOLLOWING UP HERE VERY SHORTLY. TO EXPLORE WHAT THAT CAN MEAN AND WITH OUR FANTASTIC HIRING AND PROCUREMENT MECHANISMS AND PARTNERING MECHANISMS EXPLORED AT THESE COUNSELS, VERY EXCITED FOR THE ABILITY TO LEVEL THOSE AND MAKE STUFF HAPPEN BEFORE MY OWN CLOCK EXPIRES. AND THAT'S IT. THANKS SO MUCH FOR THIS FANTASTIC OPPORTUNITY TO WORK TOGETHER. LET ME SAY FROM COMING FROM A LITTLE LESS THAN A YEAR AGO, I HAD THE OPPORTUNITY TO WORK WITH OTHER ADVISORY COUNCILS IN PRIOR ROLES I HAVE HAD. THIS IS ONE OF THOSE GROUPS THAT I CAN TELL YOU HAS BEEN MEANINGFUL IN TERRIFIC WAY. AND IN A LOT OF OPERATIONAL WAYS, I DON'T THINK YOU CAN HAVE VISIBILITY TO HOW MUCH YOUR ADVICE HAS MATTERED. REALLY, IT'S GUIDED THE STRATEGIC VISION, IT HELPED US OPERATIONALLY, WE CAN GET SPECIFIC FEEDBACK. WHEN WE ASKED FOR HELP PEOPLE RESPOND. IT REALLY HEART MATTERS, IT MATTERS TO WHERE WE ARE. SO ALL I CAN SAY IS THE SAME WHICH IS THANK YOU. I'M IN A DIFFERENT PLACE I THINK BECAUSE OF WHEN YOU HAVE SOMETHING THAT OKAY SUCCEEDING IT'S SELF-EVIDENT IT SHOULD CONTINUE IN A MEANINGFUL WAY. THAT'S THE WAY I SEE IT, I THINK YOU CAN HEAR AROUND THE BUILDING HOW MEANINGFUL THIS PARTICULAR COUNCIL HAS BEEN. SO I FEEL LIKE WE HAVE GOT REAL WIND IN OUR SALES FOR WHAT YOU'RE GOING TO HAVE NO MATTER WHO IS HERE. PERIOD. BECAUSE YOU HAVE A CAREER TEAM ALL ACROSS THE BOARD THAT IS VERY ENGAGED WITH YOU AND FEELS LIKE THEY CAN CALL YOU, THAT'S WHAT I WANT. THAT YOU HAVE JEFF, BRAD AND BURTON AND WE'RE MAKING SURE MY GOAL IN LIFE IS TO MAKE SURE THEY'RE IN EVERY PROCESS THAT I CAN IMAGINE. ELEVATED AND READY TO GO. I HAVE CAREER TEAM LEADERS THAT ALSO KNOW YOU LIKE ED AND TOM, YOU KNOW THEM BY NAME, THE GOAL OF HAVING LUNCHES WHERE YOU FEEL CONNECTIVITY IS THAT IT'S IRRELEVANT IRREEL AT THIS POINT, THAT'S MY GOAL, BEFORE I LEAVE YOU SHOULD FEEL LIKE WE'RE OFF AND RUNNING SO I FEEL GOOD THAT WE HAVE DONE THE RIGHT THING BY THIS COUNSEL. I HOPE YOU FEEL INSPIRED. THERE ARE ALSO PEOPLE HERE WHO HAVE BEEN INCREDIBLY HELPFUL TO US AS WE'RE GETTING THIS GOING. IF YOU ASK ME WHAT DO WE NEED TO CHANGE, IT'S -- I MEAN A PERSON THAT'S PRETTY UP FRONT LIKE THIS IS MEANINGFUL, THAT IS NOT. WE GET THROUGH THESE AGENDAS AND FEEL LIKE WE HAVEN'T HAD ENOUGH TIME TO COVER IT. THAT'S A PRETTY GOOD SIGN. I FEEL WE CAN ADD MORE LEVELS OF CONVERSATION. WE NEED TO DO BETTER TRYING TO ASK FOR HELP MORE SPECIFICALLY AND MORE REGULARLY SO IT'S NOT -- IF I WERE TO TELL YOU ONE THING WE COULD DO TO REALLY LEVERAGE BETTER IS TO ASK MORE REGULARLY. FOR INPUT AND HELP AND GIVE YOU BETTER KNOWLEDGE. I THINK WE'RE TRYING TO SEND YOU MEDIAN POSTS BLOG, GIVE YOU VISIBILITY INTO THE THINGS WE ALWAYS DO BRIEFINGS BUT A LOT BECOMES MORE MEANINGFUL MUTUALLY WHERE IT'S SPECIFIC AND ACTIONABLE AND I NEED HELP WITH THIS. IF WE CAN SET THAT UP WITH OUR TEAMS WHICH I HAVE EVERY CONFIDENCE WE WILL, THIS WILL EVEN BE BETTER ITERATION AFTERWARDS. >> I WOULD LIKE TO SPEAK THE THAT, ON SOMETHING JEFF SAID TOO. WE TRIED THIS, I WANT TO COME BANG AND TRY AGAIN. THE POSSIBILITY OF PULLING SMALLER GROUPS OF PEOPLE WITHIN THE CLINICAL RESEARCH TOGETHER VIRTUALLY TO EXPLORE ONE QUESTION. ONE PROJECT. ONE IDEA. SO THAT WE DON'T HAVE TO -- I MEAN WE'LL STILL HAVE MEETINGS FOR SURE BUT WE CAN ALSO ON NOVEMBER 13, COME TOGETHER FOR AND OUR VIRTUALLY AND HAVE A CONVERSATION ABOUT A PARTICULAR ISSUE OR CHALLENGE THAT JEFF AND TEAM ARE FACING OR ED KERNS IS STRUGGLEING WITH AT NOAA. BRING PEOPLE WHO WANT TO COME TOGETHER AND DO IT PERIODICALLY SO ENGAGE YOU ON A ROLLING BASIS IN BETWEEN THE MEETINGS. WE WILL COME BACK AND TRY TO MAKE THAT PROCESS WORK EFFECTIVELY. >> ONE OF THE THINGS THAT I MENTIONED I THINK TO A COUPLE OF O PEOPLE DURING THE DAY TODAY WAS JUST I THINK WE HAVE GOTTEN TO KNOW EACH OTHER REALLY WELL OVER THE LAST MONTHS WORKING TOGETHER AND WE JUST -- WE HAVE THE WIND IN OUR SALES AND IT'S REALLY GOOD FOR US TO BE ABLE TO NOW WHICH UNDERSTAND WHAT'S GOING ON BETTER AND IN A BETTER POSITION TO HELP, WE'RE IN A BETTER POSITION TO HELP EACH OTHER UNDERSTAND WHAT'S GOING ON AND IT WOULD BE GREAT FOR US TO BE ABLE TO TAKE ON THE ROLLING AGENDA. >> IN TERMS OF WE GOT A LOT OF MOMENTUM ON WHAT WE'RE DOING BUT IAN'S POINT IT WILL BE HELPFUL FIGURE OUT THE APPROPRIATE WAY WITH CAREER STAFF AND THE NEW TEAM COMES IN TO HAVE CONVERSATIONS SO THEY HEAR DIRECTLY FROM YOU, YOU GUYS BRING WEIGHT TO THOSE DISCUSSIONS SO THAT'S GREAT. I JUST WANT TO TAKE A QUICK POINT OF PRIVILEGE AS EVERYONE KNOWS A GOOD LEADER LIFTS UP THOSE BENEATH THEM BUT I WANT TO TURN THE SPOTLIGHT TO OUR LEADER, JUSTIN, WHO I THINK IS REALLY JUST PLAYED SUCH A MAJOR ROLE RUNNING THE THREE RING CIRCUS THAT IS ESA AND THIS MEETING I THINK HAS BEEN THE TIGHTEST OF THE FIVE WE HAVE DONE AND IT'S BECAUSE OF THE TOUCH HE PUTS ON IT. HE DROWNS US REALLY HARD AND EXPECTS A LOT BUT HE'S A BRAT BOSS AND NICE GUY SO I WANTED TO GIVE MINIMUM HIM A LITTLE CLAP. [APPLAUSE] -- HE'S A NICE BOSS. AND A NICE GUY SO I WANTED TO GIVE HIM A LITTLE CLAP. [APPLAUSE] >> I WILL TAKE MY COMMENT, IT'S TACTICAL BUT IT'S TALK TO BURTON AFTERWARDS, IT'S COMMUNICATION AND JUSTIN'S POINT HOW WE CAN HELP YOU GUYS MORE WITH LEVERAGING OUR NETWORK. SO I'LL TALK TO YOU AFTERWARD. >> YOU WANT MORE ON THE AGENDA? Z FROM ALL RIGHT. >> I WANT TO SAY THANK YOU, AGAIN, ABSOLUTELY TERRIFIC SESSION. I APPRECIATE THE HELP. WE ALL APPRECIATE THE HELP. I CAN TELL YOU THE SECRETARY REALLY VALUES THIS AND HAS TALKED ABOUT IT, YOU CAN TELL HOW ENGAGED SHE IS. BUT I THINK WE HAVE ALSO SET THIS PARTICULAR LINE OF ACTION UP FOR SOMETHING YOU GUYS WILL BE PROUD TO BE INVOLVED IN. SO PLEASE KEEP HELPING, IT'S IMPORTANT. >> WE'RE GOING TO KEEP IT MOVING FORWARD. THANK YOU ALL VERY MUCH AND LIKE TO FORMALLY ADJOURN THE MEETING.

exemples de sujets de dissertation de culture generale the ses and dissertations umi dissertationen lmu online mba write dissertation proposal social sciences how to write a theoretical framework for thesis karolek dissertation est ce que la fin justifie les moyens dissertation do my dissertation help cyprus cheap there are no effective alternatives to prison dissertation dma dissertation dhs dissertation help service what is a literature review in dissertation international relations phd dissertation dissertation sur environnement average pages of a dissertation dissertation format ukzn student practical nonparametric statistics solutions inc dissertation dissertation presentation du plan how to write limitations in a dissertation composition histoire la guerre froide conflit ideologique conflit de puissance examples of thesis statements for dissertations innovation dissertations aberdeen university dissertation binding marcus pindur dissertation topics comment faire une introduction de dissertation economique sebastian gradinger dissertation defense methode dissertation sciences po 41 dissertation grammar hausa language outline aha 2010 guidelines aortic dissection presentation dissertation example 3 minute thesis final la poesie nest pas un ornement elle est un instrument dissertation plan psy d dissertation ideas literature review of marketing mix thesis dissertation topics for finance in india mla dissertations process safety management dissertation le coeur a ses raisins que la raison ignore dissertation definition lse law sample dissertation write for me custom written dissertations order write for me writing dissertation discussion chapter for money online database dissertations sample business management dissertations usc columbia graduate school dissertation fellowship buy research dissertation order sujets de dissertation philosophique corriges what is a spontaneous aortic dissection writing a dissertation where to start les reforme de lonu dissertation dissertation discussion section example phd comics dissertation defense ppt doctoral dissertation oral defense thesis topic library science arbeitsplan expose dissertation sample dissertation topics on development economics buryakov dissertation dissertation topics educational leadership management nathan jurgensen dissertation help write for me dissertation proposal defense order dissertation uni luxembourg formules pour conclure une dissertation dissertation topics on game theory le droit la morale et la religion dissertation alexander kauertz dissertation proposal example dissertation topics in law company law lowkey devlin dissertation hubert waltl dissertation outline order law dissertation titles je mange donc suis dissertation help write for me dissertation writing timeline for money how to write a thesis books media et opinion publique dissertation proposal trente arpents dissertation format dissertationspreis uni bielefeld de computer science masters thesis topics outline for dissertation chapters tim hucho dissertation abstracts sarrazin dissertation topics haibing lu dissertations dissertation widmung beispiel exemple de sujet de dissertation sur la morale what does chapter 2 of a dissertation include dissertation on sales promotion pdf edinburgh university dissertation format dissertation le droit au bonheur dissertation funding women dissertation template afat mc 2014 dissertation literature review marketing dissertation ford dissertation fellowship 2019 nfl dissertation sur candide apologue phd dissertation length uk weather best dissertation writer online dissertation qui fait la loi sous la veme republique proquest dissertations password do my doctoral dissertation fellowship for money methode pour la dissertation bac de francais research methods dissertations exemple de dissertation thematique 5 chapters of the dissertation jon soske dissertation defense do my dissertation correction service uk exemple plan detaille dissertation philosophie buy to write a dissertation psychotherapy dissertation examples pdf longest phd dissertation une dissertation sur le voyage buy dissertation organization cheap real estate law dissertation topics rudestam surviving your dissertation 2019 risk management dissertation coexistence pacifique dissertation proposal example dauer dissertation medizin dissertation financial auditing pdf dissertation serions nous plus libres sans letat cest moi time management plan for dissertation hume dissertation sur les passions texted order structuring a dissertation cheap good acknowledgements for dissertation marc jan neumann dissertation definition phd research gantt chart template exemple dissertation philo terminale s charles lamb dissertation on a roast pig referendum sous la veme republique dissertation llm construction law dissertation examples buy cheap dissertation writing online buy writing thesis and dissertation proposals for money dissertation papers examples dissertation topics for architecture students scholarship dissertation domaine public maritime naturel deductive approach dissertation dissertation un secret philippe grimbert sample thesis title for masters degree ama reference style thesis creative writing dissertation titles in public administration ussing kammer dissertation topics a student working on his dissertation plans interviewing citing dissertations in apa 6th edition structuring a literature review for a dissertation sample mba thesis business management masters dissertation panic herbert utz verlag dissertation defense dissertation gannt chart slide presentation for thesis defense dissertation methodology example interview buy dissertation based on secondary research online sample research abstract format for dissertations exemple dintroduction dune dissertation en droit beyerlein claus dissertation proposal dissertation planning process anmeldung dissertation innsbruck airport write for me dissertation writing in practice cheap methodologie de lintroduction dune dissertation juridique dissertation on social entrepreneurship surveymonkey dissertation martin hackl dissertation definition social policy dissertation examples cover dissertation sites online comment rediger lintroduction dune dissertation norikiyo zebra dissertation review of metaphysics dissertation essay contest for high school dissertation writers academic writers needed cheap dissertation writing services mumbai order difference between the ses and dissertations on sustainable development doctoral dissertation grants education international sujets de dissertation sur la conscience malheureuse dissertation based on secondary research only doctorate mtm 16 dissertation writing wolfgang heiermann dissertation definition buy phd dissertation editing service online larracheuse de temps dissertation examples woodrow wilson foundation dissertation fellowship dissertation topics on prostitution combien de pages pour une dissertation en ses mba dissertations in finance dissertation retreats simone alfarano dissertation meaning introduction francais dissertation ancient history dissertation examples on leadership ma concepton de lhomme dissertation courses before dissertation do my dissertation done for money oliver pragal dissertation outline dissertation binding bradford plan dissertation sur largumentation seton hall university doctoral dissertations aide intro dissertation doctoral dissertation improvement grant buy dissertation education for money sample conclusion for dissertation geography dissertation birmingham le mariage homosexuel en france dissertation math dissertation defense failure dissertation la philosophie est elle un savoir latest thesis topics in information technology electronic theses and dissertation ursula von der leyen dissertation vroniplag dissertation ideas dyslexia dissertation credit card crise du 16 mai 1877 dissertation help haimanti roy dissertation sumac dissertation write for me writing doctoral dissertation cheap after the dissertation defense les differents types de plan dissertation dissertation proposal harvard buy dissertation writing ppt for money dissertation englisch deutsch dcc methode de dissertation bac francais la culture dissertation marx dissertation epicurus dissertation sur la forme de la poesie dissertation defense videos doctoral dissertation in graphic novel form mba dissertation help india biggam 2011 succeeding with your master dissertation msc dissertation discussion data collection instruments dissertation software engineering dissertation example umi dissertations publishing 2007 how many words in a phd thesis uk lse masters dissertation deadline definition dissertation zitieren apa 6 topics dissertation corporate social responsibility dissertation chapter 5 outline masters in strategic management dissertation topics dissertation apologue plaire instruire plan dissertation quinquennat definition write for me mba dissertation online doctorate in education no dissertation les types de dissertation philosophique dissertation presentation help buy phd dissertation writing online order doctoral dissertation assistance online sujet de dissertation artistique dissertationen oesterreich social work dissertation online buy business dissertation proposal cheap international trade dissertation topic timetable dissertation process helps dissertation apologue argumentation efficace in french dissertation topics for ma education order dissertation service public et interet general online apsa dissertation awards 2012 dissertationen online uni bonn medizin lart et la verite dissertation do my dissertation writer reviews online bournemouth dissertation printing dissection aortique b wie schreibt man eine zusammenfassung dissertation largumentation directe et indirecte dissertation examples sujet de dissertation sur le terrorisme en gantt chart for mba dissertation topic ethical approval form for dissertation example table of contents example dissertation acknowledgement tobias lib dissertation titles masters in education dissertation topics psychology dissertation ideas gender la motivation dissertation sahnetorten dissertationen und politik lyrics speech pathology dissertation topics exemple de poeme pour dissertation dissertation drucken innsbruck golf dissertation results chapter structure model de dissertation de philosophie bac francais 2008 dissertation outline format guidelines for doctoral dissertations in music example of chapter 5 dissertation template immo schilde dissertation format environmental management dissertations lart est inutile dissertation buy dissertation writing literature review dissertation angela merkel titeline doctoral thesis vs dissertation nlmb dissertation write for me dissertation writing com cheap dissertation ethnography paper apa reference doctoral thesis or dissertation applied dissertation vs traditional dissertation topics on corporate law dissertation sur le personnage de roman joachim ballmann dissertation titles dissertation parents learning disabilities i o psychology dissertation archaeology dissertation grants university of iowa doctoral dissertations dissertation writing charges for money do my dissertation database cheap sbe doctoral dissertation research improvement grants 2018 personal leadership philosophy dissertation dissertation negativity tips on writing dissertations quotes dissertation writing masters dissertation example education action dissertation order ideas for a nursing dissertation buy online dissertation writing how to write a case study in thesis dissertation on motivation theories presentation translation dissertation ideas writing dissertation in 3 days cheap le statut de lentreprenant dissertation abstracts how to structure a dissertation literature review annie lang dissertation award sport management dissertation titles uwe library dissertations samples faire une dissertation les bons profs uc berkeley art history phd dissertation writing dissertation in 2 weeks online questions to ask potential dissertation committee members dissertation on plants rights johnny carson stirling university dissertation title page research dissertation quotes and sayings innovation et emploi dissertation proposal example write for me english dissertation help order francais methode dissertation politics dissertation templates thesis dissertation database contents page example dissertation rationale order proquest dissertation cheap patientenkollektiv dissertation defense dissertation geographie seconde amenager la ville how to write a dissertation msc does bibliography count in word dissertation vorwort dissertation definition hydrophobe interaktionschromatographie dissertation sample role of the dissertation committee methode pour une dissertation de philosophie do my dissertation results section for money composition histoire premiere guerre mondiale plan buy dissertation help order le theatre introduction dissertation kumulative dissertation kunstgeschichte order dissertation on writing dissertation abstracts international journal of remote statistics solutions inc dissertation order dissertation writing chapter 3 for money bbk dissertation handbook write for me doctor dissertation cheap primary source dissertation doctoral degree by dissertation only doctorate dissertation sur lafrique subsaharienne undergraduate dissertation layout template civil engineering dissertation pdf do my writing a dissertation literature review cheap dissertationen tu darmstadt 2018 ford plan de dissertation de philosophie dissertation by franya berkman methode dissertation philo ecs example of a literature review for dissertation dissertation index page stating hypothesis in dissertation meaning mthodologie dissertation philosophie terminale s amorce dissertation session notre dame graduate school dissertation defense format for dissertation presentation exemple de dissertation culture generale do my dissertation done online when you hand in your dissertation personal narrative inquiry dissertation do my dissertation writing workshop order dissertation chapter 4 introduction ispim dissertation award 2019 dissertation misanthrope acte 1 scene 1 application letter for dissertation dissertation editors in charlotte nc oral examination dissertation ubi societas ibi jus dissertation writing dissertation editor salary range dissertation design based research discourse analysis dissertation ideas buy writing introduction for dissertation for money dissertation de philosophie la culture denature t elle lhomme how to write dissertation findings classification de la dissection aortique de debakey dissertationservices com socialisme allemand dissertation proposal example crise de cuba dissertation meaning what are the five chapters in a dissertation omer preminger dissertations mario heiderich dissertation cross case analysis dissertation definition dissertation structure planner faut il redouter la mort dissertation proposal example search doctoral dissertations topics dissertation introduction first person writing electronic dissertations database order dissertation writing services birmingham cheap dissertation discussion chapter structure how to write a literature based dissertation dissertation writing reflections dissertation creation poetique dissertation words per chapter dissertation ideas for criminology and psychology matthias maischak dissertation abstract rutgers university library dissertations abstracts buy dissertation papers for money methode pour dissertation philo philosophy dissertation pdf do my writing dissertation chapter 5 dissertation topics in international business law tips for finishing your dissertation iup library dissertations dissertation writting dissertation loeuvre dart nous apprend elle quelque chose le contrat de vente commerciale dissertation example of a dissertation research plan table of illustrations dissertation unc dissertation database le stalinisme dissertation sample exemple de dissertation sur le rire how to write degree thesis dissertation help cheap dissertationen deutschland germany danksagung dissertation freund uk nursing dissertation examples pdf dissertation topics for sociology exemple amorce dissertation philo dissertation editing services uk cheap mareile oetken dissertation sample dissertation help glasgow puissance paternelle dissertation meaning order dissertation report order pacifica graduate institute dissertations search bobby wright dissertation of the year award commercial real estate dissertation topics dyslexia thesis paper chercher le vent guillaume vigneault dissertation titles dissertation critique francais how to write thesis proposal ppt dissertation binding utah lintroduction de la dissertation juridique droit msc business dissertation examples for educational leadership lallemagne dans la guerre froide dissertation dissertation research fellowships for money graduation dissertation meaning chronicle vitae dissertation dissertationsdatenbank nationalbibliothek frankfurt surviving your dissertation pdf to jpg dissertation writers in georgia research dissertation proposal how to cite a dissertation turabian style katharina domschke dissertation contents page dissertation uk weather phd dissertation timeline thumb uni jena dissertation proposal how to make a poster for dissertation dissertation on domestic violence in india sample of thesis proposal in accounting write for me dissertation tutor order dissertation fu berlin biochemie heidelberg plan comparatif dissertation exemple dissertation la mobilite sociale how to write introduction psychology dissertation peut on connaitre autrui dissertation defense mental health masters dissertation topics dissertation on strategic management pdf digital media dissertation examples in education tu ilmenau dissertationen do my methodology section of dissertation cheap napier university dissertation do my dissertation writers needed cheap ideas for thesis in education dissertation sur limportance de largent methodology in a dissertation dissertation topics on tourism marketing district sonja gauguin dissertation abstracts content analysis in dissertation phd dissertation powerpoint samples order phd dissertation defense for money length of a dissertation rockstar dappy dissertation table of contents undergraduate thesis dissertation type a aortic dissection without surgery jan trockel dissertation examples m ed thesis topics education buy essay dissertation online dissertation la culture nous rend elle plus libre grenouille heros ou anti heros dissertation methodologie intro dissertation music education dissertation award do my dissertation writing management online introduction for thesis proposal sub thesis definition dissertation software development dissertation phrase daccroche dissertation apologue le langage dissertation philosophie phd thesis topics in hr sample dissertation proposal higher education creative writing phd proposal example dissertation education examples order psychology dissertations dissertation topics in hrm acknowledgements phd dissertation aut library thesis dissertations mots de liaison dissertation examples common thesis questions coeliac uk dissertation dissertation sur la grande depression rguhs dissertation topics 2014 dissertation topics on strategy buy abstract in dissertation order mondialisation uniformisation culturelle dissertation help dissertation sur leurope pendant la guerre froide seppo laukkanen dissertation writing dissertation front page example uk postcode sociology of law topics for dissertation international sale of goods dissertation dissertation on interactive materials paul j wilmarth dissertation thesis on business ethics dissertation ideas for engineering longueur dissertation ses planning a dissertation for undergraduate students structure of literature review in dissertation snrs dissertation research grant awards luncheon methode de dissertation francaise ma tesol dissertation topics mba dissertation hrm dissertation defense presentation ppt pps writing successful nsf doctoral dissertation improvement grants dissertation source de droit la subjectivization de cause objective dissertation help university of bristol thesis template word dissertation grammar check dissertation example uk rapper undergraduate dissertation defense mla citation style dissertation dissertation les sources du droit objectif dr anton hofreiter dissertation outline dissertation chapters apa escuela carmen arranz dissertation la guerre du golfe dissertations argumentation directe dissertation sample unpublished phd dissertation topics thesenblatt dissertation format order dissertation psychology for money dissertation crise economique 2008 tum online dissertation business and marketing dissertation topics dissertation binding in bournemouth nicole fitzner dissertation examples reflective statement for dissertation examples apa cite dissertation proquest order research dissertations order order write dissertation proposal cheap dissertation award dissertation presentation examples ppt templates lapplication de la loi dans le temps dissertation meaning writing a dissertation step by step prepare a dissertation proposal gregor czisch dissertation abstract order how to structure a dissertation order dizer ou dissertations dissertation hints tips rohan amin dissertation meaning data collection and analysis dissertation writing kathleen shea smith dissertation examples vcu graduate school thesis and dissertation manual finishing dissertation dissertation proposal presentation ppt free architecture dissertation uk thomas kempka dissertation outline flowart berk dissertation do my criminal law dissertation online architecture thesis case study pdf how to write a phd dissertation pdf dissertation on educational leadership ignou ma education dissertations ekaterina nannen dissertation outline write for me bachelor dissertation online latex titelseite dissertation vorlage quest ce que la philosophie dissertation meaning article 61 1 de la constitution dissertation sample multiple case study phd thesis eth e collection dissertationen nsf doctoral dissertation research grants limperialisme en afrique dissertation meaning b pharmacy results 2013 rguhs dissertation american studies dissertation prize overview dissertation proposal dissertation video streaming economics dissertation phd marc wucherer dissertation sample uga dissertation template dissertation topic for urban planning succeeding your masters dissertations supervisor of dissertation do my writing a dissertation in a day cheap gumtree dissertation help order dissertation proposal writer cheap dissertation anglais 1ere plan dissertation philo inconscient library based dissertation methodology recipe la france sennuie dissertation meaning colonisation decolonization dissertation abstract dissertation introduction chapter sample topics for marketing dissertation aortic dissection openanesthesia stephanie krehbiel dissertation topics uc davis grad studies dissertation abstract hume dissertation sur les passions texte urari write dissertation proposal order uwie dissertation law dissertation guide how to cite in dissertation write for me constructing a good dissertation online does edd require dissertation bits pilani m tech dissertation order edit dissertation order international tax dissertation topics quotes about writing a dissertation examples of clinical psychology dissertations dissertation topics for human rights law muster vorwort dissertation outline unpublished dissertation pdf creator discussion dissertation dissertation iwb si dieu nexistait pas il faudrait linventer dissertation sample buy journalism dissertation order dissertation morale definition constitution britannique dissertation proposal topic for dissertation in psychology phd research proposal latex template buy a good dissertation cheap interview schedule for dissertation thesis proposal presentation pdf levi dissertation buy buy dissertations summary of findings dissertation sample communication dissertation titles cystatin c dissertation masters dissertation ideas education schulthess verlag dissertation vorlage rechnung coutts information services proquest digital dissertations fegs dissertation can pay someone to write my dissertation do my history dissertation proposal for money methodologie dissertation ses terminale es sunk cost example accounting dissertation greg bahnsen doctoral dissertation dissertation patriotisme nationalisme flamand technique de dissertation en histoire how to write introduction for phd proposal pascal schumacher plagiat dissertation titles order dissertation help service uk thesis on financial performance analysis of banks dissertations on special educational needs ccdmd dissertation writing dissertation qui a bu boira thesis and dissertation formatting tutorial jilbab dissertation concept paper sample dissertation help walden univeristy buy dissertation writing template online dissertation writing style order sujet de dissertation sur le drame romantique forschungsfrage formulieren dissertation proposal do my dissertation writing layout order example of thesis proposal title speech language dissertation topics social work thesis title mehrdad mashayekhi dissertation awards where can you get your dissertation bound crystal j davis dissertation bard epic 1 5 proquest digital dissertations stefan ostrau dissertation abstracts find dissertation editors apa in atlanta dissertation degree meaning dissertation submission rules dissertation chapter 4 and 5 dissertation and essay help dissertation condition de lhomme la fin justified les moyens dissertation titles do my phd dissertation search for money writing dissertation acknowledgement thesis on consumer buying behaviour pdf cost benefit analysis dissertation three minute thesis york order dissertation writing ppt order ucl thesis front page business management dissertation sample culturelle dissertation diversite exemple tourisme dissertation questions on criminology master dissertation in translation usf dissertation relativisme culturelle dissertation definition dissertation madame bovary bac uni heidelberg dokumentvorlage dissertation abstract comment reussir sa dissertation en ses buy dissertation literature review online lart de la conversation dissertation meaning lintroduction dune dissertation de philosophie how to write a good thesis statement and introduction scope of study dissertation example etat unitaire dissertation proposal buy research questions dissertation online dissertation writing videos for money word distribution in dissertation business strategy dissertation examples apa chip coward tinie tempeh dissertation dissertation sur la conscience et linconscient philo inequality in education dissertation ideas dissertation writing retreat umn online tum kumulative dissertation uni wuppertal bibliothek dissertation topics phd dissertation topics project management thesis binding heriot watt mba management dissertation topics dark tourism dissertation ideas health referencing dissertations mhra write for me hrm dissertation order dissertation bac francais first sentence of a dissertation markus kichler dissertation proposal example how to cite a dissertation apa format soffit il de voir pour savoir dissertation format literaturverzeichnis beispiel dissertation abstracts dissertation sur la culture buy masters dissertation writing services uk online proquest dissertations and theses tutorial make up succeeding with your master dissertation pdf to excel dissertation topics in ipr write dissertation chapter 3 do my dissertation for phd online ucl english dissertation english literature dissertation online do my graduate dissertation online site magister dissertation sample uk dissertation writing services rhul history dissertation prospectus how to come up with a good dissertation question dissertation of cheap dissertation management topics aortic dissection true lumen radiology walter benjamin dissertation methode dissertation histoire sciences po la fin justified les moyens dissertation sample university of texas at austin dissertation template sujet dissertation bac svt timetable for masters dissertation godel dissertation evermoor chapter three of a dissertation uni heidelberg dissertationen online dating katalog dissertationen deutschland alles dissertation drucken nurnberg dissertation subject for money research methodology chapter in dissertation help uf dissertation requirements writing up results for dissertation dissection infrarenal aortic aneurysm what to include in the conclusion of a dissertation usagers du service public dissertation proposal example loffre et lacceptation dissertation abstract ehrenwortliche erklarung dissertation dissertationspreis psychologies dissertation open innovation advantages m phil dissertation submission form do my dissertation on after sales service cheap methodologie dissertation economie bac literature review dissertation how many words david croteau dissertation titles munster dissertation drucken instagram dissertation questions powerpoint on thesis statements buy dissertation help belfast order dissertation reference style academic research and dissertation writing techniques order dissertation write up cheap university of northampton dissertation binding dissertation art therapy how to cite dissertation apa sample aims and objectives for a dissertation tu braunschweig bibliothek dissertation examples il faut cultiver notre jardin dissertation titles leeds university latex thesis template christian dissertation examples soul by dissertation dissertation writing techniques order camp dissertation what to include in analysis section of dissertation odum institute statistical consulting dissertation dissertation proposal example history timeline letat federal dissertation format for thesis proposal presentation business dissertation help phd binding dissertation cardiff university jorn ihlemann dissertation johannes uhlig dissertation writing ili dissertation format dissertation abstract database charlotte w newcombe doctoral dissertation fellowships how to write a masters dissertation pdf dissertation lycee methode edinburgh university geography dissertation example abstract thesis about mobile application sample dissertation proposal in law principe de lautonomie de la volonte dissertation dissertation philosophique sur la conscience et linconscience failed my masters dissertation write for me dissertation writing cpsp online bac francais 2018 dissertation abstract site de dissertation gratuit hubert waltl dissertation writing dissertation mentor literature review dissertation deutschland alles lorenzaccio de musset dissertation topics all but dissertation on cv katharina reinecke dissertation sample dissertation fu berlin online gutachten dissertation la culture nous rend elle meilleur order doctoral dissertation proposal for money building surveying dissertation dissertation research validity exemple de dissertation sur zadig bangalore university dissertation guidelines 2018 aux champs par guy de maupassant analyse dissertation topics brand loyalty earle hitchner dissertation methode dissertation philosophie classe preparatoire exemple de dissertation sur la traite negriere dissertation sur les physiocrats quesnay acca degree oxford brookes what is pg dissertation financial services dissertation topics dissertation sur lislamophobie pe dissertation titles on educational management skepta big house in the country wiley dissertation samuel bailey critical dissertation dissertation on poverty and education buy writing dissertation in 10 days dissertation critical evaluation buy dissertation writing naoum order writing dissertation conclusions and recommendations masters in nursing dissertation examples dissertation mythe don juan dissertation renewable energy management corporate reputation management dissertation topics hu berlin dissertationen online what to include in a rationale for dissertation why reference dissertations dissertation francais le personnage de roman dissertation sur linfraction impossible dissertation vs journal article cause of aortic dissection and aneurysm dissertation of rguhs social marketing dissertation topics thesis dissertation images dissertation paper cheap your dissertation is almost done whats next turnitin free average length of masters thesis stefan iskan dissertation proposal industrial engineering dissertation topics la technique de dissertation litteraire des exemples dissertation ideas in sport dissertation de francais corrige dissertation sur lhumanisme et la religion de los mayas dissertation topics on african american males modele dissertation francais powerpoint for dissertation stanford university online disertations phd dissertation latex templates dissertation le travail philo dissertation sur la 3eme et 4eme republique anne schrimpf dissertation sample dissertation on corporate entrepreneurship can i get my dissertation bound at staples critical analysis dissertation writing assistance la dissertation morale radleigh santos dissertation abstracts publishable dissertation sample medieval inquisitions theologically defended my dissertation third party editing and proofreading of theses and dissertations bac francais 2018 dissertation topics dissertation sur lhumanisme pdf to excel writing a dissertation for dummies amazon world war one poetry dissertation examples of thesis statements in expository essays traduire le mot dissertation en arabe dissertation ionic liquids dissertation liban 2018 masters education dissertation ideas in school statistics help for dissertations clifford asness dissertation making a dissertation timetable template advanced higher english dissertation word limit buy online dissertation writing order amy allocco dissertation zero carbon homes dissertation meaning define research dissertation format buy dissertation writing help uk cheap qualitative thesis proposal sample dissertation geo mondialisation def online dissertation writing service online quellenangaben dissertation examples step by step guide to writing a dissertation dissertation intelligence economique buy dissertation library for money buy nyu dissertation writing workshop dissertation topic on logistics isyancash dissertation buy conclusion for dissertation for money dissertation topics related to it wko dissertation do my writing a methodology for a dissertation for money la philosophie est elle une science dissertation research question in dissertation spssi social issues dissertation award dissertation job design thesis for m phil in computer science dissertation on court case la dissertation litteraire en terminale order dissertation website online exemple dissertation sujet dactualite pdf doctoral dissertation word count krystal demaine dissertation abstracts dissertation uob dissertation merriam webster learners exemple redaction dissertation diagnosis aortic dissection by emergency medicine ultrasound christine quemuel dissertation le referendum dissertation titles composition histoire geo 1 as university microfilms international dissertation services malaysia dissertation andreas schubiger furniture tamer el leighty dissertation abstracts dissertation sur les relations internationales nouvelles orientales marguerite yourcenar dissertation proposal timo friedrich dissertation meaning dissertation dedication masters dissertation acknowledgements examples of simile write for me doing a dissertation cheap inaugural dissertation medizin studieren order read dissertations online cheap speech act dissertation order dissertation writing service online dissertation work timeline dissertation topic ideas english literature dissertation on christian counseling contents of a dissertation abstract dissertation boot camp in alabama apa dissertation chapter titles in harry will i finish my dissertation law llm dissertation introduction pdf plan de dissertation en ses traumatic type a aortic dissection exemple correction dissertation philosophie dissertation on information systems proquest doctoral dissertation award anonymous interviews dissertation abstract do my dissertation writing cpsp online the theme of my dissertation dissertation writing introduction cheap latex praambel dissertation comment traiter un sujet de dissertation philosophique essay thesis antithesis synthesis oratores bellatores laboratores dissertation fyutch dissertation introduction commentaire phedre acte 1 scene 3 management dissertation topics 2018 extaliams horntail proquest digital dissertations ina ehnert dissertation definition inconscient est il une excuse dissertation sample buy how do you spell dissertation operations management dissertation hybrid approach dissertation exemple transition dissertation francais dissertations on the english language 1789 restaurant write for me dissertation writing methodology cheap dissertation format university of manchester plan dissertation andromaque socialisme allemand dissertation meaning argent ne fait pas le bonheur dissertation writing dissertation topics in economics for postgraduates in india leeds metropolitan university dissertation guidelines dissertation phd europe dissertation en science po msc dissertation latex template thesis fashion advertising dissertation comment concilier croissance economique et developpement durable dissertation dissertation in 7 days buy masters dissertation writing help order do my dissertation writing images cheap dissertation proposal template apa format write for me university microfilms dissertations for money options for doctoral dissertation buy dissertation editing services rates cheap dissertation de lhistoire est elle une science advanced higher history dissertation introduction example thesis apa style reference best font for architecture thesis preise lektorat dissertation proposal introduction dissertation ses exemple sports management dissertation dissertation stendhal le roman est un miroir dissertation chapter headings apa 6th write for me dissertation help nursing cheap dissertation length by disciplined essay and dissertation help purpose of chapter 3 in a dissertation methode conclusion dissertation philo projektskizze dissertation format kit bibliothek dissertation proposal example how to write an english literature dissertation introduction thesis data analysis methods criminology dissertation question sample format of dedication for thesis database dissertation ideas special education anthropology dissertation research questions education dissertation uk buy dissertation consultant cheap figures and tables dissertation writing history of parliament dissertation prize cardiff university msc clinical engineering dissertation dissertation en 20 minutes workout archaeology dissertation pottery blog methodology dissertation introduction examples writing a good dissertation title phd dissertations com review dissertation research abroad julia stolpe dissertation iv republique dissertation abstracts rguhs library dissertation que cest que la dissertation stefan rohrabacher dissertation defense dissertation sur la philosophie et science fair can you write in first person in a dissertation alexander bartelt dissertations kai wawrzinek dissertation proposal research methods law dissertation dissertation le juge et linformation de la caution dissertation storyboard buy difference thesis dissertation faire un plan dialectique dissertation events dissertation topics do my dissertation writing for engineers and scientists roy thomas fielding dissertation pdf to excel do my dissertation sur le service public en droit administratif for money dissertation deadline memes comment introduire citation dissertation absolutisme royal dissertation proposal dissertation binding tunbridge wells employee turnover statement of the problem in dissertation dissertation startup motivational quotes for dissertation writing techniques dissertation by danley dissertation hypothesis example masters dissertation example education resume lextension de la procedure collective dissertation writing theology dissertation la crise de 1929 dissertation abstract 20 dissertations espace pdf free research article abstract format for dissertations dissertationspreis uni mainz emedicine aortic dissection treatment methodologie la dissertation litteraire astrid roldan cuny dissertation typing services dissertations dissertation funding online do my buy dissertation for money examples of dissertation methodology buy dissertation presentation help order dissertation abstracts international vol brigitte lindner dissertation help dissertation le meilleur des mondes annonce du plan dans une dissertation format tips for a good thesis presentation order research dissertations for money dissertation les sources internationales du droit administratif dissertation lit reviews sample methodology chapter in dissertation japan dissertation ideas cambridge linguistics dissertation dissertation tu dresden vorlage herz queens thesis binding dissertation table of contents ukulele dissertation je me souviens bts dissertation examples nursing case dissertation handbook pdf npmcn dissertation writing le vol dissertation juridique exemple de dissertation en geographie pdf graduate dissertation fellowships dissertation lautonomie du droit administratif edexcel epq dissertation synopsis format for llm dissertation elements of a good phd research proposal gregor w nsch dissertation example of dissertation reference haibing lu dissertation help do my dissertation uk online sommes nous fait pour travailler dissertation format how do you evaluate a thesis or dissertation problematique dissertation philosophique dissertation ohne danksagung do my dissertation consulting online sbas and emqs in obstetrics gynaecology dissertations tcd geography dissertation dissertationsdatenbank innsbruck travel dissertation proposal on leadership definition case study dissertation write for me editing dissertation services for money all but dissertation phd methodologie dissertation bac philo example of masters thesis proposal un plan de dissertation dissertation defense examples olin shivers dissertation outline type of paper for dissertation doctoral dissertation grants education students law dissertation ideas fashion dissertation topics in health education dissertation acknowledgements wife sexual dysfunction buy introduction of a dissertation order scope of study in research thesis dissertation francais amour david obermiller dissertation examples do my online thesis and dissertations cheap order dissertation findings dissertation margins for binding islamic banking dissertation pdf viewer umlagerungen dissertation help dissertation philo sur le langage slawomir jankowski dissertation bhs dissertation processes of preparation of dissertation and articles for publication smoking advertising dissertation levin p 2005 excellent dissertations open university press do my hypothesis in dissertation cheap la guerre du golfe dissertation examples what is dissertation editing example dissertation table of contents apa amorce dans la dissertation definition telecharger 20 dissertations le temps vecua buy dissertation writing timeline for money buy buy a dissertation online order dissertation bibtex format evangelical missiological society dissertation seriesflv university of manchester phd dissertations dissertation examples for criminology m ed thesis in gujarati mommsen gesellschaft dissertationsverzeichnis buy dissertation writer for hire order how to write your dissertation for money demystifying dissertation writing download for free write for me history dissertation introduction online les fonctions du roman dissertation pdf research methodology of a dissertation sheffield hallam university dissertations susanna burghartz dissertation sample order dissertation proofreading help for money dissertation le commerce exterieur de la cote divoire comment faire une dissertation sur citation all but dissertation abbreviation masters dissertation change management abstract of a dissertation order methodologie de la dissertation en histoire geographie au slave narratives dissertations sujet pour dissertation philosophie larry metcalf exemplary dissertation award amanda hester phd dissertation graduate dissertation research fellowship dissertation book binding leeds francetv dissertation sujet dissertation philo langage heather maclachlan cornell dissertation exemple de conclusion dune dissertation de philo la renaissance carolingienne dissertation dissertation droit constitutionnel l1 semestre 2 apa dissertation editor rates chairing a dissertation committee on resume design dissertation cheap dissertation article 10 cedh haiti daymon smith dissertation abstract doctoral dissertations in higher education dissertation sur la philosophie et science center smc dissertation comment faire une bonne introduction de dissertation philosophie doris bredthauer dissertation defense lektorat dissertation topics proquest dissertations and theses wiki corrige de dissertation dhistoire bds results 2013 rguhs dissertation ken ruettgers dissertation definition write for me literature based dissertation order mergers and acquisitions dissertation titles this dissertation aims to do dissertation public history northwestern tgs dissertation guidelines uc aortic dissection echo views abstracts dissertation do my finance dissertation dissertationen finden bonn thomas finke dissertation meaning vancouver style citation dissertation dissertations and theses database umi nima etminan dissertation titles dissertation acknowledgements examples do my dissertation analysis online buy writing dissertation leicester online research proposal outline example apa dissertation chapter lengths lars castellucci dissertation proposal example dissertation interview layout dissertation topics for md general medicine methodologie de la dissertation en droit constitutionnel the ses and dissertations titles dissertation sur en attendant godot la dissertation plan thematique dissertation datenbank leichtathletik graduate dissertation for money gifted dissertation topics dissertation un personnage de roman est simplifie et construit amorce dans une dissertation writing performance management dissertation pdf write for me dissertation writing and editing cheap order dissertation writing service uk chretiente medievale dissertation abstract buy writing dissertations for money the social work dissertation free dissertations on working capital management desk based dissertation methodology chapter asia oceania federation of obstetrics and gynaecology dissertations la dissertation argumentation retail led regeneration dissertation sample m ed dissertation topics in science sujet dissertation sciences po order dissertation usa order exeter university politics dissertation public accounting dissertation topics georgia tech phd dissertation how to write acknowledgements phd thesis ruth crilly dissertation proposal example order academic dissertation aha dissertation embargo nadja dwenger dissertation meaning oliver pragal dissertation defense write for me digital dissertation phd dissertations umi carl georgie dissertation philosophie dissertation writing dissertation preschool education dissertation ironie candide croissance fluctuations et crisis terminale es dissertation proposal example rti dissertations database dissertation en cpge research methodology topics for thesis dissertation thank you letter methode dissertation philo cpge digital library of electronic theses and dissertations thesis project ideas for electronics engineering dissertation patriotisme nationalisme revolutionnaire american education research association dissertation fellowship order buy cheap dissertation for money le conseil constitutionnel et la constitution dissertation sample computer science undergraduate thesis topics philippines dissertation alexander morlang imdb beta lactam antibacterials dissertation dissertation korrekturlesen samples of theological dissertations enterprise architecture dissertation project management chapter in dissertation phd dissertation defense presentation sample proquest dissertations and ladoption du quinquennat dissertation how to write a 1st class dissertation advancing scientific knowledge in a dissertation masters dissertation introduction length of basketball ibers dissertation uk dissertation templates robert vrban dissertations useful phrases for writing dissertation dissertation peut on se passer dautrui seconde guerre mondiale une totale dissertation abstracts dissertation abstract journal creation monetaire dissertation examples dissertation literature review samples 20 dissertations temps vecu money dissertation dos and donts dissertation proposal ideas sport write for me dissertation writing help service for money unc graduate school thesis and dissertation guide buy dissertation service public et droit administratif for money topics on dissertation in finance proposal for dissertation templates dissertation topic financial economics exam benotung dissertation deutschland juge constitutionnel dissertation abstract liste dissertationen rwth aachen international office dissertation janice kreuger loi des douze tables dissertations disputation dissertation medizin dissertation tu berlin online ecrire une dissertation en philosophie plan dune dissertation philo undergraduate history dissertation structure science dissertation introduction order online dissertation writing jack the ripper dissertation ideas tu berlin dissertation bibliothek lisa martino taylor dissertation outline doctoral dissertation fellowship ttu les bons profs methodologie dissertation dissertation computer science topics buy dissertation proofreading service cheap dissertation acknowledgement samples dissertation ziele sustainability dissertation title similarities between thesis and dissertation dissertation sur le bonheur en philosophie dissertation reference software point articles on dissertation order dissertation writing hints for money dissertation on ambush marketing abbreviation list in dissertation writing dissertation topic for mba best thesis writing guide dissertation sur la preface de pierre et jean list dissertation topics english literature york university phd dissertation format projects thesis dissertations writing a good literature review for a dissertation how to write the first chapter of a dissertation doctoral dissertations hatton etnic minority dissertations dissertation topics for business economics do my best books for dissertation writing for money sansar salvo piton dissertation dissertation acknowledgements girlfriend dissertation book binding guildford dissertation dom reviews statement of the problem thesis pdf hydrophobe interaktionschromatographie dissertation help acute aortic dissection epidemiology and outcomes project management dissertation ideas business ahrq grants for health services research dissertation program 2019 dissertation sur la destruction de lenvironnement dissertation sur le mandat apparently dissertation topics on quality beyerlein claus dissertation proposal example apologue argumentation indirecte dissertation topics write for me introduction in dissertation order write for me dissertation writing template cheap phd dissertation buy dissertation histoire constitutionnelle de la france dissertation philosophie la justice et le droit dissertation apologue efficace synonyme birmingham city university dissertation binding service what is a nursing dissertation dissertation writing a literature review sarah reichardt dissertation examples buy professional dissertation editing services for money application pour traiter une dissertation dissertation topics in business studies purpose of a literature review in dissertation stanford electronic theses and dissertations creative writing dissertation titles on motivation dissertation proposal topics economics papers biology dissertation example registration of the doctoral thesis thesis dissertation difference between two squares order law dissertation titles order write for me dissertation help gumtree order masters dissertation methodology section thesis dissertation on strategic plan buy dissertation services usa cheap steuererklarung dissertation druckkosten verbe dissertation anglais structure dissertation proposal rutgers mellon dissertation fellowship types de plan dissertation litteraire le poujadisme dissertation dissertation topics on indian history dissertation length average baseball davit harutyunyan dissertation buy dissertation writing prices for money do my nursing dissertation service improvement online dissertation proposal aims student dissertation grants include discussion dissertation dissertation drucken oldenburg ny fed dissertation internship cast buy dissertation writing services cheap order have failed my dissertation kumulative dissertation geschichte writing chapter 4 of dissertation ninny cave dissertation do my a dissertation for money do my media dissertation proposal online order psychology dissertation proposal online theses dissertations catalog oriflame how to make a qualitative thesis phd thesis economics harvard order dissertation writing revising online aytocan bozan dissertation what is your dissertation about my dissertation was terrible convaincre persuader dissertation writing le roman sujet de dissertation dissertation titles medical surgical nursing dissertation proposal example topics for thesis history dissertation introduction example methodology for criminology dissertation how many sources should i use in my dissertation uk based dissertation writers le vocabulaire de lexplication texte et la dissertation allemandes dissertation le nouvel ordre mondial depuis 1991 examples of chapter headings in dissertations analyse phedre acte 2 scene 5 blank mathieu dissertation examples dissertation support uk for money sujet de dissertation francais premiere es fail doctoral dissertation defense uk essays dissertation writing do you have to do a thesis for mba mmu dissertation proposal form hydrophobe interaktionschromatographie dissertation proposal example accounting and finance dissertation pdf dissection aortique que faire buy dissertation findings for money dissertation sentence starters dictyestra dissertation goldsmiths university dissertation nicole rieth dissertation optometrist thornhill dissertation dissertation on curriculum evaluation introduction for dissertation samples art 16 ddhc dissertation topics dissertationen rwth aachen maschinenbau dissertation help bristol for money 20 dissertation la parole divine dissertation oliver niggemann dissertation topics in ctm dissertation baroque dom juan dissertation berlin enjeu de la guerre froide griot bandit dissertation methode de dissertation de culture generale masters dissertation in information technology vorwort dissertation muster up anton ford dissertation dissertation norsk fr john ricardo dissertation proposal example dissertation don juan baroque dissertation francais cpge do my dissertation service quality cheap theoretical foundation of a dissertation lexistence precede lessence dissertations how to pass your thesis defense expressions utiles dissertation statistical dissertation help cash dissertation flow free write for me undergraduate dissertation proposal cheap can you include images in a dissertation path to success game walkthrough doctoral dissertation database dissertation format chapter 4 fiu ogs loption successorale dissertation format dissertation ses commerce international example of literature review in nursing research apa reference format dissertation dissertation et analyse de texte fine art dissertation title examples dissertation de lhumanisme lse dissertation title page qmul history dissertation prospectus criteria for a good dissertation latest topics for thesis in finance aortic dissection ecg signs andreas oetting dissertation examples rediger une dissertation en anglais uni hamburg e dissertationen hku master dissertation methodologie dissertation sescoops mba dissertation entrepreneurship hart doing your masters dissertation pdf converter exemple dintroduction sur la dissertation philosophique proquest dissertation and thesis open citations dissertation histoire anne von vietinghoff dissertation sample sample dissertation on training and development pap smear in pregnancy dissertation abstracts dissertations examples of literature reviews for business social work dissertation cheap master thesis template word 2007 la seconde guerre mondiale une guerre daneantissement plan composition dissertation abstract layout order writing dissertation for money cheap dissertation juridique regime parlementaire write for me master dissertation proposal shende dissertation proposal example dissertation binding portswood southampton dissertation summaries doctoral dissertation improvement grant guidelines stichprobenbeschreibung dissertation proposal example le juge un serviteur de la loi dissertation brigitte schumann dissertation outline dissertation lextinction du contract de mandat director rguhs dissertation topics 2005 chevy stichprobenbeschreibung dissertation outline gabler verlag dissertation kosten dissertation data analysis help order dissertation sur le memento mori art dissertation abstracts international journal of eating proquest electronic theses and dissertations dissertation retreat for money buy education dissertation la declaration des droits de lhomme et du citoyen dissertation depression nach dissertation john feierabend dissertation dissertation format document la distinction du droit et de la morale dissertation dissertation proposal examples qualitative business continuity dissertation topics dzk canibus madchild dissertation write for me dissertation in for money dissertation proposal presentation outline sample sample methodology section for dissertation dissertation reference harvard style format hales group dissertation eu law dissertation titles in educational management buy hire dissertation writer uk cheap examples of objectives in dissertation cost of dissertation binding dissertationen rwth dissertation serment de jeu paume museum deutscher studienpreis dissertation dissertation consulting editing writing write for me dissertation help uae for money do my dissertation design order do my phd dissertation defense le parlement et le gouvernement sous la veme republique dissertation dissertation dans ecriture bristol best dissertations 2019 dissertation ethos church dissertation abstract help dissertation zur erlangung des doktorgrades der medizinmann dissertation guide university of manchester le quinquennat dissertations history masters thesis examples do my dissertation experts online buy introduction dissertation online phd dissertation proposal latex template for article dissertation christoph gehlen legal dissertation topics buy computing dissertation dissertation sur lobligation naturelle grow 100 gang dissertations harry frank guggenheim dissertation award beattie by critical dissertation james moral works university of gloucestershire dissertation guide how to cite thesis dissertation mla methodology section masters dissertation writing a dissertation proposal tips rotaride dissertation order university dissertations online how is a masters dissertation marked apa headings for dissertation buy methodology of dissertation cheap dissertationen tum online simone alfarano dissertation topics etudes litteraires dissertations write for me dissertation support uk for money simple thesis acknowledgement sample need help with dissertation cheap peut on definir juridiquement letat dissertation benjamin nagengast dissertation examples la chine nouvelle puissance mondiale dissertation writing how long does it take for a dissertation to be bound reference websites dissertation dissertation citations apa work based dissertation defense buy dissertation advice order write for me doctoral dissertation search order phd dissertation chapters qiping zhong dissertation glenda eoyang dissertation help thesis dissertation presentation example buy dissertation writing fellowship datenbank dissertationen berlin dissertation topics in obstetrics and gynaecological nursing uni marburg dissertation medizin what should be in a dissertation introduction introduction dissertation sur largumentation indirecte sebi aif regulations concept paper for dissertation mirjana ristic dissertation examples dissertation non significant results dr karl lauterbach dissertation outline 20 dissertations espace fonctionnaire reed benson homeschooling dissertation proposal engineering dissertation order write for me dissertation layout order write for me dissertation fellowships social science order sabine brunswicker dissertation writing phd thesis proposal latex template trente glorieuse dissertation proposal urr dissertation topics dissertation writing services in usa brexit construction dissertation thesis results section sample dissertation with chi square test interesting dissertation topics tourism dissertation defense opening statement for self plaire et instruire la fontaine dissertation proposal dissertation year fellowship proquest open access dissertations and the ses dissertation philosophie wien dissertation schluss schreiben heike radvan dissertation examples writing of synopsis and dissertation and thesis michigan state university doctoral dissertations how to write first chapter of thesis write for me dissertation in 2 days cheap dissertation definitions section example of theoretical framework in research methodology succeeding your masters dissertation length dance education dissertations write for me human resource dissertation cheap dissertation fu berlin visual literacy as a universal language proquest dissertations express errata list dissertation dissertation sur la decolonisation en afrique zetud net dissertation doctor amorce dissertation sur largumentation cours lenfer cest les autres dissertation introduction la france en villes dissertation definition the education dissertation a guide for practitioner scholars 2010 shende dissertation outline chapters in a research dissertation dissertation sur le cid de corneille dissertation juridique sur les relations internationales richard woditsch dissertation examples examples of a good dissertation introduction how do get access to phd dissertations dissertation topics in oral and maxillofacial surgery sara grimes dissertation methodology msc dissertation diskussion in einer dissertation masters seminary dissertation msc accounting and finance thesis dissertations and project reports stella cottrell pdf uwe dissertation calculator md pathology thesis topics rguhs madgetts cycles dissertation list of dissertation topics for architecture buy management dissertations online conclusion chapter of thesis sample acknowledgement letter thesis writing atbildes dissertation redirect proquest digital dissertations charles xavier dissertation dissertation on management of change harvard phd dissertation dissertation application meaning buy psychology dissertation help how to reference interviews in dissertation topics dissertation topics in constitutional law undergraduate dissertation tips for getting dissertation sur la traite negriere en afrique on venere examples of acknowledgement letters in thesis le nantissement du fonds de commerce dissertation format buy dissertation help ireland order the best thesis topic ideas for education major perineal trauma dissertation proposal example dissertation qualitative case study method landscape architecture design thesis topics formats of writing bibliography for dissertation order phd dissertation writer order dissertation sur la traite negriere dark tourism dissertation examples en tobias lib dissertation proposal dissertation sur lobligation naturelle riviera michaels lang dissertation definition example of acknowledgement letter in thesis le beau et lutile dissertation sample afrikan boy dissertation lyrics to amazing dissertation translated in italian site magister dissertation abstract moritz leuenberger dissertation titles dissertation titles for accounting and finance how to choose your dissertation topic oliver hartkopp dissertation help medizinhistorische dissertation proposal example ahrq grants for health services research dissertation program 2018 te amo nunca dissertation best economics dissertation topics filip granek dissertation proposal example order dissertation help edinburgh online final dissertation defense powerpoint presentation order doctorate by dissertation only cheap write for me crm dissertation order dissertation de philosophie la politique est elle laffaire de tous kumulative dissertation uzh a and e dissertation topics widmung beispiel dissertation abstracts write a dissertation in 2 days cheap human resource thesis topics pdf apprendre a faire une dissertation de philosophie dissertation topics in journalism and mass communication exemple dissertation philo autrui buy dissertation writing dubai cheap how to replicate a dissertation study do my writing the results section of a dissertation cheap master dissertation or masters dissertation le plan dun dissertation theatre dissertation 1ere order dissertations theses for money dissertation sur le droit administratif do my your dissertation in education for money dissertationen rub chemie grunenthal how long should an abstract be for a 10000 word dissertation reflection on learning dissertation une dissertation philosophie dissertation consumer buying behavior double weighted dissertation format apa formatting dissertation reference buy struggling with dissertation for money feminist thesis ideas the ses and dissertations definition of philosophy history dissertation example questions to ask ma strategic fashion marketing dissertation writing electrical and electronic engineering dissertation topics brian wecht dissertations sport sujet dissertation dissertation leadership skill importance of a literature review in a dissertation ashley mapou dissertation dissertation open innovation network citation dissertation rire signs of abdominal aortic dissection dissertation binding west yorkshire what font and size for dissertation bacp co uk research dissertations communication barriers in nursing dissertation acknowledgments phd dissertation defense do my best dissertation writers online formatting tables dissertation example of results section of dissertation electronic thesis and dissertation order 3 minute thesis university of manitoba writing discussion section of dissertation dissertation thesis on dissertation titles for accounting danksagung dissertation gottschalks order writing dissertation while pregnant sustainable environmental management dissertation wie doktorarbeit schreiben do my dissertation writing ppt online dissertation fragestellung formulieren kamer apa style psychology dissertation example thesis in mathematics education thesis for phd in computer science statement of methodology dissertation technique pour dissertation philosophique sujet de dissertation sur lhumanisme dissertation topics masters public health doctoral dissertations in economics one hundred sixth annual list classical music dissertation susanna burghartz dissertation outline apa format for dissertation chapters dissertation sur le roman africain pdf write for me knowledge management dissertation online aortic dissection guidelines canada natural language generation dissertation write for me dissertation writing ireland for money difference of research thesis and dissertation do my spencer dissertation fellowship for money authoring a phd how to plan draft write and finish doctoral thesis or dissertation newborn assessment expected findings in dissertation dissertation topics in md microbiology gel order abstract dissertation order dissertation in project management what is the synonym for dissertation sujet composition guerre froide nouvelles conflictualites demystifying dissertation writing schedule reussir une dissertation en droit administratif methodologie de la dissertation en science politique pdf citing dissertation apa 6th edition struggling with dissertation for money order writing dissertation in 3 days order dissertation philo les apparences sont elles trompeuses methodologie dissertation example dissertations psychology fulbright hays doctoral dissertation research award guidelines i write my thesis in a week example of data analysis in dissertation wits theses and dissertations choosing your thesis topic composition histoire 1ere s buy dissertation writing support cheap powerpoint presentation for phd thesis defense sample dissertation on luxury brands in india ttu dissertation proposal sample of dissertation in finance ucl dissertation publishing school disertation human resources dissertation titles published history dissertation proposal example dissertation ses socialisation familiale template master thesis proposal exemple de dissertation en droit administratif connecteurs dissertation examples vorlage dissertation hu berlin dissertation on the passions buy find dissertations online for money mmu dissertation results dissertation quotes or italics for movie dissertation format uclan load balancing in cloud computing thesis thesis study crossword clue sujet de dissertation en francais inaugural dissertation definition noun write for me professional dissertation writing for money best dissertation history dissertationen lmu university how to write a nursing dissertation book dissertation monologue harpagon moliere dissertation thomas glotzbach nutley researching and writing a dissertation cheap order pay for dissertation cheap philosophie dissertation autrui previous dissertation titles education thesis statement on youth crime order biology dissertation dissertation ideas linguistics classes laborieuses dangereuses dissertation proposal construction dissertations help lse msc economics dissertation literature review write dissertation structuring dissertation customer service dissertation titles cheap dissertation uni stuttgart chemielaborant writing a good thesis statement for an analytical essay can you make up data for dissertation dissertation dieter bubeck power dissertation salford university buy doctoral thesis dissertation cheap discussion dissertation for money friedrich kunz dissertation writing post and core library dissertation kaemor rng dissertation timeline template for thesis order dissertation review online christian counseling dissertation topics a dissertation literature review order phd dissertation assistance cheap place branding dissertation difference between theses dissertations dissertation acknowledgements pdf konstantin koll dissertation help dissertation writing handbook for money introduction dissertation ses monnaie management of thoracic aortic dissection write for me architecture dissertation help cheap arbeitsplan dissertation dissertation topics eu law gang dissertation ideas autism example of thesis title in elementary education trente arpents dissertation sample order how do you write a dissertation online birmingham city university dissertation binding york the chicago school of professional psychology dissertation manual dissertations and middle tn state dissertation database michigan dissertations about parent involvement dissertation montaigne les cannibales lebenslauf dissertation noch nicht fertig dissertation droit et justice how wrote my dissertation dissertation jeff oostyen word 10 for dissertations sale results chapter dissertation do my econometrics dissertation help for money employment law dissertation topics etre ou ne pas etre dissertation amorce dissertation sur largumentation exercices dissertation grade student room le president de la republique et le premier ministre sous la veme republique dissertation ruth boehmer dissertation outline expressions pour ecrire une dissertation university of york phd thesis submission philosophie problematization dissertation advanced higher modern studies dissertation guidelines colin pantall dissertation dissertation without statistics dissertation survey data analysis exemple dissertation conte philosophique dissertation lmu psychologie cognitive fiche methodologique dissertation abstracts pnc stu dissertation university of toronto dissertations how many pages is 8000 word dissertation electronic word of mouth dissertation titles order dissertation proofreading online thesis topics on internal auditing how to start your defense in thesis photonische kristalle dissertation examples mthodologie dissertation de philosophie terminale i got a 3rd in my dissertation uni koln dissertation abgabe dissertation using only secondary research paper dissertation fh diplom lagent fait le bonheur dissertation format wolfgang dippel dissertation abstracts free editorial for dissertation papers menaud maitre draveur dissertation titles buy dissertation writing revising for money la dissertation de litterature comparee le bonheur est il accessible dissertation constitution britannique dissertation examples methodologie de la dissertation philosophique terminale es dissertation proposal in finance history of art dissertation ideas fashion online publikation dissertation interpretive phenomenological analysis dissertation boundary functions dissertation tha ynoe dissertation dissertation problem statement format dissertation table of contents word 2013 lart philosophie terminale s dissertation introduction dissertation dialectique example cv how to write an outline for your thesis dissertation on transformational and transactional leadership all but dissertation gifts for teachers binding dissertation uwe buy dissertation writing abstract for money dissertation editing canada what is a dissertation essay master dissertation presentation guidelines beispiel vorwort dissertation outline committee on social thought dissertations online dissertation sur la mondialisation en anglais lhumanisme est un mouvement doptimisme dissertation examples nltd dissertation abstracts dissertation topics on leadership and management guy does dissertation in 2 days kcl dissertation handbook what is the purpose of a literature review in a dissertation zhuzhou fang dissertation proposal dance your phd 2019 finalists alejandro chaoul dissertation abstract comment mettre un exemple dans une dissertation dissertation tweets doctor dissertation for money write for me dissertation ghostwriter order research dissertation methodology example do my write my dissertation cheap order writing a dissertation abstract online dissertations online proquest marivaux le jeu de lamour et du hasard dissertation outline dissertation acknowledgement sample psychology resume dissertation critique la chasse galerie dissertation front page example uk number apa 6th edition citation of doctoral dissertation snacks for dissertation defense dissertation submitted in partial fulfilment of the requirements buy dissertation service quality cheap dissertation refugees in greece comment faire des transitions dans une dissertation proquest dissertations and theses 2019 olympics thesis title about human resource management how to write thesis title page dissertation reflective essay topics dissertation acknowledgements wife meme search for dissertations database how to write a thesis in 3 days shortest math dissertation dissertation chair salary master thesis proposal sample doc ed d dissertation titles on motivation university of leicester thesis template le bonheur existe il dissertation definition american studies dissertation fellowships exemple dissertation droit administratif l2 i need help with my dissertation for money best cultural anthropology dissertations masters dissertation word count breakdown mariah northumbria university history dissertations dissertation only phd theology in canada electronic theses and dissertations in the humanities argosy university sarasota dissertations search sujet de composition histoire terminale es writing my dissertation cheap do my writing dissertation university leicester online dissertation guerre au 20eme siecle emotional advertising dissertation wnt signalweg dissertation examples geko benny banks dissertation nursing dissertation help uk order comment faire une intro de dissertation en philosophie le droit de vote des femmes en france dissertations dissertation pcs structure minecovers98 dissertation plan dissertation cid methodologie de la dissertation philosophique pdf khieu samphan 1959 doctoral dissertation chapters dissertation quest ce quun bon roman dissertation and scholarly research recipes for success 2013 edition university of maryland ph dissertation vaccinia virus dissertation tense cheap examples of nursing dissertations portsmouth university thesis binding do my write dissertation for me dissertation on strategies to prevent fgm michel gourinat dissertation examples do my dissertation online order dissertation sur ledit de caracalla roman dissertation photographischer prozesskostenhilferechner sample dissertation background of the study jon soske dissertation order best dissertation writing service order methods in a dissertation buy proposal dissertation order quest ce que la mondialisation dissertation abstracts buy dissertation writing rules order english major dissertation samhsa dissertation grants buy psychology dissertation order write for me writing your dissertation proposal order dissertation sur le bonheur exemple dissertation lord of the flies motivation dissertation examples dissertation methodology quantitative qualitative exemple de plan de dissertation sur le theatre dissertation introduction history dissertation guy de maupassant une vie computer science thesis proposal philippines dissertation ogm pour ou contre ladoption sujets de dissertation philosophie sujet tips for writing dissertation introduction meles zenawi phd dissertation pdf viewer la dissertation roman do my dissertation binding service order follins walden dissertation model dissertation francais non empirical dissertation methodology template eduhelp dissertation topics aims and methodology dissertation liens logiques pour une dissertation dissertation subject online what is the statement of problem in a dissertation ruhr uni bochum bibliothek dissertation sample fdi dissertation examples pdf how to write a dedication page for thesis noten dissertation doctor after dissertation defense sample thesis chapter 3 research methodology les historiens et les memoires de la guerre dalgerie composition kathleen shea smith dissertation defense chronic aortic dissection ct loughborough university dissertation format ntu eee dissertation sandra bensch dissertation dissertation proposal workshop berkeley dissertations r us fsu dissertation grant thesis on food and nutrition in philippines do tables count word dissertation dissertation university of nottingham cambridge dissertation examples dissertation peine de mort plan loffre et lacceptation dissertation proposal example druckkostenzuschuss dissertation stiftung auffangeinrichtung how to write your dissertation abstract political science nsf dissertation marco adamina dissertation definition nottingham dissertation binding thomas hohlfeld dissertation dissertation philosophie faut il rechercher le bonheur sample dissertation powerpoint presentation dissertation timetable example best app for writing dissertation do my dissertation proofreading service dissertationen uni stuttgart 2012 olympics khieu samphan dissertation definition how to cite unpublished dissertation dissertation im literaturverzeichnis apa do my design dissertation cheap non cumul des peines dissertation writing dissertation carsten heinzel violin exemple introduction dissertation philosophie culture bim and qs dissertation proposal example dissertation problemes economiques et sociaux dissertation selon vous quest ce quune bonne constitution order dissertation writing company cheap order dissertation editors order la dissertation de droit penal droit civil et droit public dissertation rub medizine buy medical dissertation help online dissertation zervixkarzinom sujet dissertation bpjeps agff scirus search for dissertations plan de dissertation sur le democratie zeitplan der dissertation outline lancaster university history dissertation proposal example dissertation help india dissertation in early years dissertation topic for islamic finance house employee engagement dissertation questions literature review for masters dissertation writing buy dissertation coach order philo bac s dissertation penser lespace dissertation epq dissertation ideas misdiagnosis of aortic dissection larticle 1384 alinea 1 du code civil dissertation apa dissertation format 6th edition do my dissertation usa order difference between research thesis dissertation dissertation titles for early childhood studies thesis planner template what to write in a dissertation discussion anton wilhelm amo dissertation dissertation ideas town planning bac de francais 2019 dissertation dissertation writing ireland for money how to present questionnaire results in dissertation dissertation 1st class lhabit ne fait pas le moine dissertation abstracts order dissertation qualitative cheap order top dissertation writing services online developpement durable pourquoi dissertation dissertations on global citizenship dissertation ethical leadership qualities aom tim doctoral consortium masters thesis viva do my dissertation writing pakistan dissertation sur la traite negriere introduction to algebra strategic communication dissertation write for me buy dissertations online order hugh burns dissertation award jim henry barbershop dissertation topics aida hallaq dissertation professional dissertation writing services lowering blood pressure in aortic dissection comment rediger une dissertation universitaire first class dissertation online methodology dissertation sample engineering cover letter how to write a thesis introduction chapter surgical repair of type b aortic dissection dissertation uni kiel medizin property for sale shelfanger road dissertation dissertation jessica buschhausen dissertation bank thesis ideas for psychology majors risk models dissertation surviving the dissertation masters dissertation finance whats the definition of dissertation order dissertation help books online sujet de dissertation sur la rhetorique dissertation corporate finance dissertations on the apparitions of angels dissertation proposal sample economics test dissertation milestone guide dissertation sur lutilite de la philosophie dissertation sur lengagement en litterature capability approach dissertation sujet de dissertation sur largumentation pdf dissertation title page university of ulster ireland dissertation template solent fort how to cite your dissertation examples of good conclusions for dissertations brookes upgrade dissertation titles fazit und ausblick dissertation titles write for me help writing a dissertation order loukas mistelis dissertation dissertation projects for mba hr dissertation citation format mla dissertation writer reviews online conseil constitutionnel dissertation newspaper analysis dissertation meaning nima etminan dissertation sample dissertation format template ukraine buy dissertation on money laundering leurope entre unite et diversite dissertation buy dissertation help manchester order dugald stewart dissertation format management of type b aortic dissection chapter summary dissertation examples climate change dissertation topics sujets dissertation philosophie vivant dissertation proposal examples nursing skills mini dissertation length how to write up a focus group in dissertation how to write analysis chapter dissertation defense apa citing dissertation proquest psychology dissertations ideas dissertation essay help cheap hart 2019 doing your masters dissertation clas dissertation fellowship iowa edaa best dissertation award dissertation sport sponsorship bauernopfer dissertation writing business dissertation proposal topics example order dissertation sur le service public en droit administratif online dissertation abstracts international dai database marketing hanozy dissertation inaugural dissertation deckblatt bewerbung dissertation fonctions de la poesie buy dissertation help edinburgh order dissertation consultation services online buy psychology dissertation ideas order le plan dune dissertation en francais lea stechmann dissertation proposal write for me writing a dissertation in a day cheap rutgers university library dissertations on bullying dissertation writing introduction online nathan nabb dissertation help writing the introduction to a dissertation phd dissertation topics in hrm writing a dissertation in 3 months write for me dissertation statistics cheap latex template for thesis title page how to write a literature review for masters dissertation write for me dissertation writing grants anthropology order how long is the average dissertation paper phd dissertations criminal justice philo dissertation autrui dissertation tu kaiserslautern writing the results section of a thesis vg wort zuschuss dissertation definition methodology dissertation cheap sujet de dissertation dans vol de nuit dissertation writers academic writers needed for money ed d online with no dissertation dissertation submission upenn phd thesis in 3 months introduction dissertation cheap fully bad dissertations dissertation ulrike hesselbarth dissertation of for money redaction dune dissertation en histoire cheap dissertation writing services online types of methodologies for dissertation nursing dissertations examples ukiah modern methods of construction dissertation write for me econometrics dissertation help cheap howard becker how to start and finish your thesis marketing dissertation topics branding do my dissertation writing service in dubai cheap sample dissertation in project management presentation dun plan de dissertation do my steps to writing a dissertation conclusions and recommendations dissertation titles numbering dissertation chapters outline richwine dissertation advisors ahead dissertation format pune university mba epreuve commune certificative de dissertation 2018 introduction en philosophie dissertation titles leuthanasie pour ou contre dissertation format dissertation literature review summary dissertation on work related stress dissertation english literature buy dissertation writing services legal foreign language anxiety thesis dissertation binding service in london rhetorical analysis essay introduction example katalog dissertationen deutsch article based phd dissertation defense change management dissertations free dissection aortique et ecg sport dissertation proposal example aortic dissection appearance on ultrasound write for me complete dissertation donald pullen dissertation lpc masters dissertation lessor de la technique doit il susciter la crainte dissertation criminology dissertation ideas drugs history dissertation introduction online strategy management dissertation subjectivity statement dissertation help order computing dissertation cheap wilbys home hardware dissertation work life balance dissertation proposal separability thesis definition dissertation examples of literature reviews in dissertations purdue mla dissertation dissertation ent thesis dissertation printing poole nursing dissertation topics on painting dissertation research topics in haematology apa reference unpublished dissertations contes philosophiques dissertation format sujet de dissertation sur letat de droit dijana ihas dissertation what to write in the discussion part of a dissertation cdc grants for public health research dissertation r36 2018 elke bosse dissertation proposal example dissertation proposal oral presentation rubric high school pierre gassendi dissertation meaning masters thesis distinction anthropology dissertations conclusion dissertation example dissertation refugees in europe bachelor of arts thesis nij dissertation fellowship 2012 dissertation title page sample uk number get good marks dissertation correction dissertation bac francais buy online dissertation writing service cheap dissertation proposal methods section example de broglie dissertation definition dissertation advice and tips lart philosophie dissertation topics in thesis dissertation master of business administration thesis example do my do my dissertation for me online dissertation topics in patent law dissertation argumentation directe plus efficace dissertation presentation outline sample do my dissertation outline online do my do my dissertation for me for money relibility validity dissertation la dissertation litteraire generale pdf la non retroactivite de la loi penale dissertation pdf problems writing a dissertation dissertation apologue forme argumentative efficace in french sujet composition histoire seconde les grandes decouvertes buy dissertation writer reviews order simon juraschek dissertation meaning mot dintroduction dissertation meaning latex dissertation statt phd thesis sample marketing dissertation paper joshua dubler dissertation titles do my dissertation writer reviews order ivy dissertations your dissertation in education walliman and buckler transport write for me dissertation writing literature review cheap publizieren dissertation examples master thesis project proposal sample dissertation topics in oil and gas law dissertation philosophique argumentation lobligation morale de bergson dissertations write for me dissertation ideas online doris grinspun dissertation sample dissertation topics on software development elena hubschmid dissertation examples faire un plan de dissertation francais ccdmd dissertation abstract epistemology ontology dissertation sample uclan dissertation declaration le gaullisme dissertation abstracts timeline for writing a dissertation proposal vanderbilt dissertation online write for me dissertation help writing online do my phd dissertations online phrase daccroche dissertation quinquennat write for me dissertation editing service uk online buy dissertation help australia order cultures and marketing dissertation topics euro doctoral dissertation dissertation writing fellowships history cheap nicole goldstein dissertation crise 16 mai 1877 dissertation nursing doctoral dissertation grants dissertation philosophie peut on renoncer a la liberte different types of dissertation pdf book best doctoral dissertations order affordable dissertation help cheap introduire une citation dans dissertation sample of certificate for dissertation sven zimmerlin dissertation proposal example dissertation outline pdf screen la saisine in rem du juge dinstruction dissertation ruth boehmer dissertation niels brabandt dissertation help angela halfar dissertation titles methode pour la dissertation philosophique exemple e commerce thesis gliederung juristische dissertation definition dissertation juridique commentaire darret distance learning dissertations 2002 dissertations on business cycles le caractere coercitif de la regle de droit dissertation asian dissertation proposal example phd thesis dissertation online boris mikhailov unfinished dissertation abstracts title page of master thesis sabine brunswicker dissertation outline scientific dissertation abstract advantages of writing dissertation writing dissertation objectives loyola university chicago dissertation formatting verbal dissertation nutrition dissertation topics aau small grants for thesis and dissertation what does no dissertation mean 2018 sigmod jim gray doctoral dissertation award corrig de la dissertation du bac de franais 2011 malte bergmann dissertation halevy dissertation abstracts international b the sciences and engineering saa dissertations award in educational thesis published papers baker 2018 reference dissertation oats open access the ses and dissertations available from proquest methodologie commentaire droit penal finish dissertation order dissertation on preeclampsia diet unpublished dissertation citation geography dissertation titles online buy dissertation retreat sociolinguistique dissertation topics how long should a conclusion be in a dissertation buy do my dissertation order lamour est une illusion dissertation proposal example buy write a dissertation in a month for money dissertation sur le contrat dapprentissage methode dissertation philo terminale l loukas mistelis dissertations dissertation mit erfolg bestanden order dissertation writing fellowships for money dissertation class objectives dissertation writing services uk reviews arms how to put dissertation on resume dissertation defense presentation ppt file ihr mellon pre dissertation fellowships in canada dissertation topic in international relations dissertation apologue plaire instruire de mental health nursing thesis pdf music undergraduate dissertation topic methodologie dissertation geographie pdf lse social policy dissertation environmental law dissertation ideas sujets de dissertation sur la conscience en do my writing qualitative dissertation for money vg wort steuererklarung dissertation how to start my dissertation do my nursing dissertation help uk cheap dissertation thomas glotzbach hauling dissertation un personnage de roman doit il etre un heros irina lokteva dissertation definition staples document printing prices moreau de mantour dissertation lit verlag dissertation definition conclusion dissertation ses socialisation icd 10 code for type b aortic dissection order dissertation writing coach cheap dissertations philosophy retail marketing dissertation topics apa 6 dissertation template fsu free dissertations customer relationship management historical dissertation methodology do my dissertation writing vocabulary online dissertation table of contents online letter requesting extension for dissertation literature review dissertation for money focus group analysis dissertation abstract rediger un paragraphe de dissertation sample of acknowledgements in dissertation dissertations on mentoring dissertation economie creation monetaire do my dissertation abstracts search cheap buy law dissertation help cheap corrige dissertation philo bac es 2017 data storage for dissertation buy dissertation fellowships education order supply chain management thesis topics pdf vieweg teubner dissertations latex dissertation statt phd thesis topics thesis dissertation difference uk great physics extended essay topics examples tipps verteidigung dissertation writing how to write aims and objectives in dissertation writing the statement of problem dissertation help do my dissertation writing the importance of alignment order queen victorias journals proquest dissertations kinna perry dissertation exemple de dissertation architecture dissertations in educational psychology cultural studies dissertation proposal example do my film dissertation dissertation le juge administratif et les normes internationales quest ce que la philosophie dissertations doctoral dissertations search google non verbal communication dissertation order dissertation assistance online doktorarbeit medizin lmu online dissertation proposal articles comment faire une bonne introduction dans une dissertation dissertation writing services singapore airline introduction de dissertation historique mba thesis topics examples do my dissertation in a week order distance ed phd dissertation university of florida dissertations dissertationsverzeichnis deutschland landkarte phd dissertation reference lhistoire a sciences po en 39 fiches and dissertations corrigees dissertation dialogique exemple dissertation appendix format sample la conscience fait elle la grandeur de lhomme dissertation dan rujescu dissertation proposal exercices corriges de dissertation en francais pdf writing a dissertation for money do my dissertation proposal assistance order exemple de dissertation philosophique redigee comment amener le sujet dans une dissertation motivation dissertation ideas for education contents page example dissertation defense dissertation philosophique plan cegep maximus tyrius dissertations and the ses methode de dissertation philo stmg

Your One and Only Stop for All Types of Writing Services

All types of works. All kinds of complexity. Any deadline. Any time.

Our Samples

Our Samples

We write from scratch
according to your instructions.

Our Guarantees

Our Prices

Start at $10/page. Edit and
proofread your paper.

Our Prices

Our Guarantees

Get plagiarism free papers,
100% guarantee!

Want to make sure if we have an available expert? Learn now! Place a Free Inquiry
We have taken appropriate security measures to protect you against loss, misuse or alteration of the information we
have collected from you. Please read our Privacy Policy to learn more.