Raja Farrukh Ali

I am a Ph.D. Candidate in Computer Science at Kansas State University, advised by William Hsu. I am a member of the KDD lab and my area of research is reinforcement learning.

Previously, I obtained an M.S. in Computer and Communication Security from SEECS NUST, where I was advised by Adnan Khalid Kiani on the design of secure and efficient wireless routing metrics. I completed my B.E. in Avionics Engineering from CAE NUST, where I worked under the guidance of Asad Amir Pirzada on hybrid wireless mesh networks.

If you would like to collaborate on a research idea, please send me an email. I am very enthusiastic about collaborations and open-source projects. I also mentor students who are seeking graduate school in the U.S, or are just excited about doing research.

Email / CV / Google Scholar / GitHub / Twitter / LinkedIn

News

2023-12: Papers accepted to AAMAS 2024, and AAAI 2024's XAI4DRL Workshop. Attended NeurIPS'23!

2023-11: Paper accepted to the NeurIPS 2023 XAI in Action Workshop.

2023-05: Congrats to Kevin Duong on being awarded Distinguished Undergraduate Student in Research award.

2023-02: At AAAI-23 to attend Doctoral Consortium and present a poster.

2023-01: Paper accepted to AAMAS 2023 as an extended abstract.

2022-12: Two papers accepted to the NeurIPS 2022 Deep RL Workshop.

2022-11: Following in the footsteps of my advisor, I have been accepted to the AAAI-2023 Doctoral Consortium!

2022-11: Paper accepted to AAAI-2023 as Student Abstract.

2022-07: Excited to be selected for the CIFAR DLRL Summer School 2022.

2022-07: Attended ICML 2022 in Baltimore, had some great conversations and an amazing time.

2022-06: Paper accepted to the DARL workshop @ ICML 2022!

2022-05: Accepted to the Oxford ML Summer School. Will be attending both the ML x Health and ML x Finance parts.

Research

I am broadly interested in designing autonomous agents that can learn complex behaviors in both cooperative and adversarial settings. I believe that with the ever-increasing presence of autonomous systems in our society, we will need to come up with algorithms and interfaces necessary to build efficient, safe, reliable and interoperable RL agents which can exist beyond individual ecosystems. In addition to this multi-agent RL perspective, I am also interested in designing agents that can learn from demonstration (reward induction, inverse RL) as well as improving the generalization capabilities of agents.

My past research focused on the design of safe and efficient routing metrics for use in adhoc wireless mesh networks. I also have 8+ years of experience working in industry as an embedded systems engineer, with some software development workload. I have also been a Graduate Teaching Assistant for 3 years at Kansas State University, where I loved teaching C language to engineers, and introductory computer science courses to CS majors.

	Policy Optimization using Horizon Regularized Advantage to Improve Generalization in Reinforcement Learning Nasik Nafi, Raja Farrukh Ali, William Hsu, Kevin Duong, Mason Vick Accepted to AAMAS 2024 We present Horizon Regularized Advantage (HRA) estimation that enables robustness to the underlying uncertainty of episode duration.
	Explainable Reinforcement Learning for Alzheimer’s Disease Progression Prediction Raja Farrukh Ali, Ayesha Farooq, Emmanuel Adeniji, John Woods, Vinny Sun, William Hsu NeurIPS 2023 XAI in Action Workshop [ Paper, Code ] How SHAP can be used to explain RL models that predict disease progression.
	Non-Exponential Reward Discounting in Reinforcement Learning Raja Farrukh Ali AAAI-2023 Doctoral Consortium [ Abstract ] The roadmap of my dissertation.
	Analyzing the Sensitivity to Policy-Value Decoupling in Deep Reinforcement Learning Generalization Nasik Nafi, Raja Farrukh Ali, William Hsu NeurIPS 2022 Deep RL Workshop, AAMAS 2023 (Extended Abstract) [ Paper, Abstract ] This work analyzes the generalization performance compared to the extent of decoupling between policy and value networks.
	Multi-Horizon Learning in Procedurally-Generated Environments for Off-Policy Reinforcement Learning Raja Farrukh Ali, Nasik Nafi, Kevin Duong, William Hsu NeurIPS 2022 Deep RL Workshop, AAAI 2023 (Student Abstract) [ Paper, Abstract ] We study the auxillary task of multi-horizon learning in procedurally generated environments (Procgen) under behavior policies enforced through exponential and hyperbolic discounting functions.
	Hyperbolically Discounted Advantage Estimation for Generalization in Reinforcement Learning Nasik Nafi, Raja Farrukh Ali, William Hsu ICML 2022 Decision Awareness in RL (DARL) Worskhop [ Paper ] We present hyperbolic discounting-based advantage estimation for policy gradient optimization in generalization tasks.
	Network Adaptive Interference Aware routing metric for hybrid Wireless Mesh Networks Ubaid Ullah, Adnan Kiani, Raja Farrukh Ali, Rizwan Ahmad IEEE IWCMC 2016 [ Paper ] NAIA selects better routes by taking into account both inter and intra-channel interference.
	Energy-Load Aware Routing Metric for Hybrid Wireless Mesh Networks Adnan Kiani, Raja Farrukh Ali, Umair Rashid IEEE VTC Spring 2015 [ Paper ] ELARM chooses best route based on link stability and network load conditions. It also evaluates link stability based on receiving node's energy conditions.
	Load Dependent Dynamic Path Selection in Multi-Radio Hybrid Wireless Mesh Networks Raja Farrukh Ali, Adnan K. Kiani, Asad Amir Pirzada IEEE WCNC 2014 [ Paper ] D-WCETT routing metric selects paths with least level of congestion by taking into consideration locally available queue information.

Last Updated: Jan 2023
Credits for the template to Jon Barron