If a group of unaffiliated individuals attack a country, that country has no recourse for nuclear retaliation.
Some governments, at least including the US, various UK states such as Canada and Australia, the PRC, and some continental European powers, have had agents working full time on just getting samples of radionucleotides from various fission plants, and analyzing those samples so that, if those nucleotides turn up in a dirty bomb or worse, an actual fission device, they can tell just where they were made by differences in various isotopic ratios, trace elements, and such. Knowing the source does not always mean those nations would retaliate against an attack from a group of apparently non-affiliated individuals, but it's certainly one piece of evidence in building a case for retaliation that would satisfy at least part of the international community. Nations have some interest therefore in reporting thefts of materials internationally, and various governments have some interest in setting up conditions for such reporting (i.e. in some cases, assuring the reporting country this will be classified and not released, and so hopefully not available for political candidate's uses.)
I'd say that against groups such as you describe, it may not be possible to respond with nuclear retaliation, or recourses may be limited. It may also be desirable to respond with something less damaging to innocent bystanders, other nations, and the environment, even if a nuclear option is possible. This could go anywhere from a use of actual boosted fission devices within hours of the first event, to a much more measured response, possibly weeks or even longer after the first event.
By the way, probably the most workable term for 'unaffiliated individuals' in US sources is "non-state actors", relatively short, straight-forward and to the point. In the US, emergency response teams called NEST would be responsible for the first stages of gathering samples from a dirty bomb incident or similar event, but their primary purpose is to stop such events before there is a detonation or risk to the public, if that's still possible when they become involved. NEST now stands for Nuclear Emergency Support Team, but in some older sources, the S stood for Search instead of Support. Calling one a NEST Team is redundant, but occasionally done by the media. NESTs are authorized to respond to incidents both inside and outside US borders, but just what that means in practice is unclear..