Virata manual

Conditions, functions and operators

The small language you type inside if, generate and replace.

What we have

  • if and in on almost every command
  • Stata's missing-value rule everywhere, including in the generated R
  • ln log log10 exp sqrt abs round int floor ceil mod min max
  • missing cond inlist inrange, and strlen substr upper lower real string
  • _n and _N, which follow the groups under bysort
  • factor notation i. c. # ##, and the time-series operators
  • Excel's SUMIFS, COUNTIFS, AVERAGEIFS, MAXIFS, MINIFS (and SUMIF, COUNTIF, AVERAGEIF), with Excel's own rules
  • pw, aw and fw weights, depending on the command

Coming later

  • bysort x (y):, which sorts within the group on the same line
  • VLOOKUP, XLOOKUP, INDEX/MATCH and IFERROR — not planned; merge does that job with the relationship checked
  • Excel's date functions (YEAR, MONTH, EOMONTH): there is no date type yet
  • runiform() and the other random-number functions
  • scalar and matrix
  • the extended macro functions, such as ` : word count ...' ``
  • r() is set by some commands and not others; e() only after a model

Choosing rows: if and in

Almost every command takes if, and it means the same thing everywhere: only the rows where this is true.

summarize mpce if sector == 1
drop if age < 15
regress mpce age if state == "Kerala"

in chooses by position instead: list in 1/10 is the first ten rows, in -5/l the last five. Position, not identity — sort the data and in 1/10 means different rows.

Note == with two equals signs for "is equal to". One = is assignment, and Virata will tell you when you have used the wrong one.

Comparing and combining

You writeIt means
== !=is equal to, is not equal to
> < >= <=greater, less, greater or equal, less or equal
&and — both must hold
|or — either will do
!not
inlist(x, 1, 3, 5)x is one of these
inrange(x, 15, 59)x is between these, ends included

Brackets group things, and it is worth using them even when they are not strictly needed: if (age >= 15 & age < 60) | retired == 1 is readable, and the same line without brackets is a guess.

The missing-value rule — read this one

In Stata, and therefore here, a missing value is treated as larger than any number. This is the single most common way to get a wrong answer without seeing an error.

drop if age > 60        <- also drops everyone whose age is blank
count if income > 1000  <- also counts everyone whose income is blank

If that is not what you want, say so:

drop if age > 60 & !missing(age)

The same rule is reproduced everywhere Virata works, including in the R it prints for you — that is what the stata_missing() helper in the generated R is for. R's own filter() drops blanks instead, so without the helper the two languages would quietly disagree.

Missing is written as a single dot: replace age = . if age > 120.

Functions you can use

Arithmetic and shaping: ln() log() exp() sqrt() abs() round() int() floor() ceil() min() max() sum().

Choosing between values: cond(test, a, b) gives a where the test holds and b where it does not — an if-then-else inside an expression.

Checking: missing(x) is true where x is blank. inlist() and inrange() as above.

The row number: _n and _N

_n is the number of the current row and _N is how many rows there are. On their own they are of modest use. Under bysort they become one of the most useful things in the language:

sort household age
by household: generate rank = _n
bysort household: generate size = _N

Inside each household, _n counts its members and _N is how many there are.

Stata lets you sort within the group on the same line, writing bysort household (age):. That form is not supported yet — sort first with sort, then use by, as above.

by and bysort

by state: summarize mpce runs the command once per state. The data must already be sorted by that variable, which is why bysort state: ... exists — it sorts and then runs, in one line. Use bysort unless you have a reason not to.

It works as a prefix on summarize, generate and egen. Stata's bysort state (age):, which sorts within the group as well, is not supported yet.

Categories in models: factor notation

regress mpce age i.sector i.state
logit poor c.age i.sector
regress mpce i.sector#i.sex

i. in front of a variable means "treat this as categories, one effect each", measured against whichever category comes first (the base level). c. forces a variable to be treated as a plain number. # between two makes them interact, so the effect of one is allowed to differ by the other.

Time-series operators

After tsset, these work in any expression:

You writeIt means
L.xlast period's value (lag); L2.x two periods back
F.xnext period's value (lead)
D.xthe change since last period: x - L.x; D2.x the change in the change
S12.xthe seasonal difference: x - L12.x, this month against the same month last year

They count in units of the time variable, not in rows. L.sales at month 5 is month 4's value, and it is missing if month 4 is absent — even when the row above happens to be month 3.

Excel's SUMIFS and its family

If you came from a spreadsheet, these work inside generate and replace:

generate dept_total = SUMIFS(salary, dept, dept)
generate n_senior   = COUNTIFS(dept, "Sales", level, ">3")

SUMIFS, COUNTIFS, AVERAGEIFS, MAXIFS and MINIFS, plus the older SUMIF, COUNTIF and AVERAGEIF spellings.

A criterion can be a fixed value ("Sales", 3), a comparison written as text (">3", "<=10", "<>Sales"), a wildcard ("S*", "?ales"), or a whole column — which means "rows like this row", the same answer egen total = sum(salary), by(dept) gives.

These add nothing egen and if could not already do. They are here because you already know how to write them.

Two rules change inside these functions

They follow Excel's rules, because the function name is what says which dialect you are in. Two of those rules disagree with the rest of Virata:

Inside SUMIFS and friendsEverywhere else
a blank, against ">3"never matchesmatches — missing is larger than any number
"sales" against "Sales"matches; capitals ignoreddoes not match

So these two lines give different answers on the same data, and both are right:

count if level > 1                    <- 5: Stata counts the blanks
generate n = COUNTIFS(level, ">1")    <- 4: Excel does not

VLOOKUP, XLOOKUP, INDEX/MATCH and IFERROR are not here and are not planned. Matching rows across two files is what merge does, with the relationship checked and with diagnostics for the trailing spaces and capitals that make a lookup fail silently.

Weights

summarize mpce [pw=multiplier]
tabulate sector [fw=households]

pw is a probability weight — this row stands for this many in the population, which is what survey files carry. fw is a frequency weight — this row *is* this many identical rows. aw is an analytic weight.

For survey data, set the design once with svyset and use the svy: commands rather than putting weights on each command. Weights alone give you the right averages and the wrong standard errors.