Ethics and Data Privacy

Artificial Intelligence (AI) has the potential to revolutionize employment counseling, offering personalized insights, efficient job matching, and innovative learning opportunities. However, its integration into the field also poses some risks and challenges, particularly because employment counseling usually collects, processes, and stores a lot of beneficiaries’ personal and sensitive data. This section specifically addresses the main risk that comes from the use of AI in employment counseling related to the data privacy concerns.

What is personal data?

The European Union’s General Data Protection Regulation (GDPR) defines personal data as “any information that relates to an identified or identifiable living individual (data subject).” Personal data also includes different pieces of information which, taken together, may lead to the identification of a specific person. This legal definition still applies even when the data has been de-identified, encrypted or pseudonymised. Such data can only be no longer categorized as personal data when it has been anonymised and the anonymisation is irreversible.

Some examples of personal data in the employment counseling context include:

Name
Date of birth
Home address
Email address contains an identifiable name such as “name.surname@company.com”
CV
Employment history
Academic records
Personality assessment
Skills test result
Internet Protocol (IP) address
Message and email exchange during job counseling

In addition, there is some data that is not allowed to be processes, including someone’s:

Racial or ethnic origin
Sexual orientation
Political opinions
Religious or philosophical beliefs
Trade union membership
Genetic, biometric or health data except in specific cases (e.g. when you've been given explicit consent or when processing is needed for reasons of substantial public interest, on the basis of EU or national law)
Personal data related to criminal convictions and offences unless this is authorised by EU or national law

Main concerns regarding personal data processing using AI system in employment counselling

Data processing refers to any action carried out on personal data. This includes collecting, recording, organizing, structuring, storing, modifying, retrieving, consulting, using, transmitting, disseminating, aligning, combining, restricting, erasing, or destroying personal data. Data processing can be done manually or digitally.

There are several concerns regarding the processing of personal data using AI system in the context of employment counseling:

1. Transparency

The use of digital tools to process personal data of a beneficiary may make the process less transparent. Beneficiaries often do not realize which data is collected and why it is processed. In addition, how an AI system in a tool processes their data is not always visible from the first glance. The AI model in the tool can feel like a black box which is hard to understand even by the employment counseling organization.

2. (Algorithmic) bias and risk of discrimination

Algorithmic bias happens when machine learning algorithms generate unfair or discriminatory results due to systematic errors. Since AI-enhanced tools are created by humans, they tend to reflect biases that humans have. Algorithmic bias can also happen through data flaws. It occurs when data is collected incorrectly or an AI model is trained using unrepresentative data that does not equally represent all groups. If the personal data is processed using a biased AI model whose result is used to make a life-altering decision, some beneficiaries may experience discrimination.

3. Security and confidentiality

Since digital tools always have the risk of being hacked, there is a constant threat to the security and confidentiality of the personal data stored and processed with the tool. Personal data can also be exposed to unauthorized parties even within the employment counseling organization itself due to careless handling. People or parties with malicious intent may also misuse the personal data that can endanger the identifiable beneficiaries.

Mitigating the risk concerning personal data

Employment counseling organizations can mitigate the risk regarding personal data processing by following some ethical actions when processing beneficiaries’ personal data.

It is important to always start personal data collection by informing the beneficiaries about the data that will be collected, the reason for the data processing, and how long they will be stored. Beneficiaries also have the right to know who is processing their data and whom to contact if they have any issue regarding the data processing.

In addition, the employment counseling service has to be able to explain how the AI tools are used in the data processing work and how it leads to specific results if necessary.

Mitigating the algorithmic bias may be difficult for employment counseling service staff, since they are also a product user and not the one responsible for developing the AI tools. However, there are some actions they can take to minimize the risk. When choosing the AI tools, employment counseling services have to study and understand the product well and confirm whether the AI tool adheres to the principles of safe and ethical AI.

Furthermore, being the parties who collect the personal data, it is important to ensure that the data is collected properly and updated regularly. By doing this, the employment counseling service can feed and train the AI model using good and representative data.

Finally, employment counseling services have to implement technical and organizational security measures to protect personal data from breaches, unauthorized access, and loss (e.g., encryption, access controls, secure storage).

They must have an emergency response plan in the case of a data breach or unintended data exposure, such as a data breach notification. In the case of a data breach, the organization must report to the supervisory authority within 72 hours if there is a risk to individuals' rights and freedoms. Affected individuals must also be informed if the breach poses a high risk to them.

Privacy rights for data subjects

In addition to the mitigation measures, employment counseling services must always respect the following rights for their data subjects/beneficiaries.

These rights are listed and protected under the GDPR.

Right to be informed: Data subjects can obtain information about the processing of their personal data.
Right of access: Data subjects can obtain access to the personal data held about them.
Right to rectification: data subjects can ask for incorrect, inaccurate or incomplete personal data to be corrected.
Right to erasure: Data subjects can request that their personal data be erased when it is no longer needed or if its processing is unlawful.
Right to restriction of processing: Data subjects can request the restriction of the processing of their personal data in specific cases.
Right to data portability: Data subjects can receive their personal data in a machine-readable format and send it to another data controller.
Right to object: Data subjects can object to the processing of their personal data for marketing purposes or on grounds relating to their particular situation.
Rights in relation to automated decision-making and profiling: Data subjects can request that decisions based on their personal data and that significantly affect them are made by natural persons, not only by computers.

Following the mitigation measures and respecting the beneficiaries’ data privacy rights are crucial to ensure an ethical AI implementation in employment counseling.

References

Kuśmierczyk, Małgorzata, Algorithmic Bias in the Light of the GDPR and the Proposed AI Act (May 8, 2022). Maciej Olejnik, Wiktoria Morawska (ed.), "(In)equality. Faces of modern Europe", Wydawnictwo Centrum Studiów Niemieckich i Europejskich im. Willy’ego Brandta, Wrocław, 2022 (forthcoming), Available at SSRN: https://ssrn.com/abstract=4117936 or http://dx.doi.org/10.2139/ssrn.4117936

General Data Protection Regulation (GDPR): This EU law governs the processing of personal data within the EU. https://commission.europa.eu/law/law-topic/data-protection/legal-framework-eu-data-protection_en#:~:text=The%20GDPR%20has%20been%20incorporated,throughout%20the%20European%20Economic%20Area

Your Europe. 2025. Data Protection Under GDPR. https://europa.eu/youreurope/business/dealing-with-customers/data-protection/data-protection-gdpr/index_en.htm

European Commission. 2025. Data Protection Explained. https://commission.europa.eu/law/law-topic/data-protection/data-protection-explained_en

GDPR. 2025. What is GDPR, the new EU law? https://gdpr.eu/what-is-gdpr/

IBM. 2024. What is algorithmic bias? https://www.ibm.com/think/topics/algorithmic-bias